2024-09-17T08:09:20.624 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T08:09:20.639 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T08:09:20.751 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908203 branch: reef description: orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} email: ceph-qa@ceph.com first_in_suite: false job_id: '7908203' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 1 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: reef ceph: conf: global: mon election default strategy: 3 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: false mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - CEPHADM_AGENT_DOWN - CEPHADM_FAILED_DAEMON log-only-match: - CEPHADM_ sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: reef sha1: 1cbdc9211df088cccbd94423c7cf3a2b85fe432b owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - osd.0 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.1 - client.1 - prometheus.a - grafana.a - node-exporter.b - - mon.c - mgr.z - osd.2 - client.2 - node-exporter.c seed: 5234 sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 sleep_before_teardown: 0 subset: 46/64 suite: orch suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_1cbdc9211df088cccbd94423c7cf3a2b85fe432b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 1cbdc9211df088cccbd94423c7cf3a2b85fe432b targets: smithi047.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJceNo/oMJQPTUTJrqIfDSk2pM2eAdayIlfZdn6b2qEi5A9h8ZuSFeYui7uhBC9qrhM+rZOwexCmNs1DNbCDZ+A= smithi193.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCiaSDetivHvqgoMXz//TPWit/bjWIkWOXRZ7qyDCyt5Hbw37stLCh3QxNJKuhXyJtcwn27iWTQDzZ8wzoa5WFY= smithi203.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPJLlBgXWoNEZxMLf2oWvyOLf/tLLM7lTRyS/2cCyEoNpIbVshnD8B4rur/YNmXxR+akSbL/YnekVnnxPcPZqvc= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install runc nvmetcli nvme-cli -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - nvme_loop: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - stat -c '%u %g' /var/log/ceph | grep '167 167' - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - ceph orch ls --format yaml - ceph orch ls | grep '^osd ' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_22:08:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T08:09:20.752 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_1cbdc9211df088cccbd94423c7cf3a2b85fe432b/qa; will attempt to use it 2024-09-17T08:09:20.752 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_1cbdc9211df088cccbd94423c7cf3a2b85fe432b/qa/tasks 2024-09-17T08:09:20.753 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T08:09:20.754 INFO:teuthology.task.internal:Checking packages... 2024-09-17T08:09:20.780 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9' 2024-09-17T08:09:20.780 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T08:09:20.780 INFO:teuthology.packaging:ref: None 2024-09-17T08:09:20.780 INFO:teuthology.packaging:tag: None 2024-09-17T08:09:20.780 INFO:teuthology.packaging:branch: reef 2024-09-17T08:09:20.780 INFO:teuthology.packaging:sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:09:20.781 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-17T08:09:21.064 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-855.g19e2a2be 2024-09-17T08:09:21.066 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T08:09:21.072 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T08:09:21.072 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T08:09:21.077 INFO:teuthology.task.internal:Saving configuration 2024-09-17T08:09:21.089 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T08:09:21.095 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T08:09:21.119 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908203', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 08:04:11.617399', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJceNo/oMJQPTUTJrqIfDSk2pM2eAdayIlfZdn6b2qEi5A9h8ZuSFeYui7uhBC9qrhM+rZOwexCmNs1DNbCDZ+A='} 2024-09-17T08:09:21.139 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908203', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 08:04:11.619903', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCiaSDetivHvqgoMXz//TPWit/bjWIkWOXRZ7qyDCyt5Hbw37stLCh3QxNJKuhXyJtcwn27iWTQDzZ8wzoa5WFY='} 2024-09-17T08:09:21.157 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908203', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 08:04:11.618751', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPJLlBgXWoNEZxMLf2oWvyOLf/tLLM7lTRyS/2cCyEoNpIbVshnD8B4rur/YNmXxR+akSbL/YnekVnnxPcPZqvc='} 2024-09-17T08:09:21.158 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T08:09:21.164 INFO:teuthology.task.internal:roles: ubuntu@smithi047.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-17T08:09:21.164 INFO:teuthology.task.internal:roles: ubuntu@smithi193.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.1', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2024-09-17T08:09:21.164 INFO:teuthology.task.internal:roles: ubuntu@smithi203.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'osd.2', 'client.2', 'node-exporter.c'] 2024-09-17T08:09:21.164 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T08:09:21.245 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f68e819fac0>, signals=[15]) 2024-09-17T08:09:21.245 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T08:09:21.251 INFO:teuthology.task.internal:Opening connections... 2024-09-17T08:09:21.251 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi047.front.sepia.ceph.com 2024-09-17T08:09:21.253 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:09:21.328 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi193.front.sepia.ceph.com 2024-09-17T08:09:21.329 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:09:21.410 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi203.front.sepia.ceph.com 2024-09-17T08:09:21.411 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:09:21.486 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T08:09:21.493 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2024-09-17T08:09:21.512 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2024-09-17T08:09:21.512 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2024-09-17T08:09:21.570 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="CentOS Stream" 2024-09-17T08:09:21.570 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="9" 2024-09-17T08:09:21.570 INFO:teuthology.orchestra.run.smithi047.stdout:ID="centos" 2024-09-17T08:09:21.570 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="rhel fedora" 2024-09-17T08:09:21.570 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="9" 2024-09-17T08:09:21.570 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el9" 2024-09-17T08:09:21.570 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T08:09:21.570 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2024-09-17T08:09:21.571 INFO:teuthology.orchestra.run.smithi047.stdout:LOGO="fedora-logo-icon" 2024-09-17T08:09:21.571 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T08:09:21.571 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://centos.org/" 2024-09-17T08:09:21.571 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T08:09:21.571 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T08:09:21.571 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T08:09:21.571 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2024-09-17T08:09:21.594 DEBUG:teuthology.orchestra.run.smithi193:> uname -m 2024-09-17T08:09:21.612 INFO:teuthology.orchestra.run.smithi193.stdout:x86_64 2024-09-17T08:09:21.613 DEBUG:teuthology.orchestra.run.smithi193:> cat /etc/os-release 2024-09-17T08:09:21.670 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="CentOS Stream" 2024-09-17T08:09:21.670 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="9" 2024-09-17T08:09:21.670 INFO:teuthology.orchestra.run.smithi193.stdout:ID="centos" 2024-09-17T08:09:21.671 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="rhel fedora" 2024-09-17T08:09:21.671 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="9" 2024-09-17T08:09:21.671 INFO:teuthology.orchestra.run.smithi193.stdout:PLATFORM_ID="platform:el9" 2024-09-17T08:09:21.671 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T08:09:21.671 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31" 2024-09-17T08:09:21.671 INFO:teuthology.orchestra.run.smithi193.stdout:LOGO="fedora-logo-icon" 2024-09-17T08:09:21.671 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T08:09:21.671 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://centos.org/" 2024-09-17T08:09:21.671 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T08:09:21.671 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T08:09:21.671 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T08:09:21.672 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server 2024-09-17T08:09:21.693 DEBUG:teuthology.orchestra.run.smithi203:> uname -m 2024-09-17T08:09:21.712 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64 2024-09-17T08:09:21.712 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release 2024-09-17T08:09:21.771 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="CentOS Stream" 2024-09-17T08:09:21.771 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="9" 2024-09-17T08:09:21.771 INFO:teuthology.orchestra.run.smithi203.stdout:ID="centos" 2024-09-17T08:09:21.771 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="rhel fedora" 2024-09-17T08:09:21.771 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="9" 2024-09-17T08:09:21.771 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el9" 2024-09-17T08:09:21.771 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T08:09:21.771 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31" 2024-09-17T08:09:21.771 INFO:teuthology.orchestra.run.smithi203.stdout:LOGO="fedora-logo-icon" 2024-09-17T08:09:21.771 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T08:09:21.771 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://centos.org/" 2024-09-17T08:09:21.771 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T08:09:21.771 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T08:09:21.772 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T08:09:21.772 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server 2024-09-17T08:09:21.793 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T08:09:21.802 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T08:09:21.808 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T08:09:21.808 DEBUG:teuthology.orchestra.run.smithi047:> test '!' -e /home/ubuntu/cephtest 2024-09-17T08:09:21.811 DEBUG:teuthology.orchestra.run.smithi193:> test '!' -e /home/ubuntu/cephtest 2024-09-17T08:09:21.813 DEBUG:teuthology.orchestra.run.smithi203:> test '!' -e /home/ubuntu/cephtest 2024-09-17T08:09:21.830 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T08:09:21.837 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T08:09:21.837 DEBUG:teuthology.orchestra.run.smithi047:> test -z $(ls -A /var/lib/ceph) 2024-09-17T08:09:21.868 DEBUG:teuthology.orchestra.run.smithi193:> test -z $(ls -A /var/lib/ceph) 2024-09-17T08:09:21.870 DEBUG:teuthology.orchestra.run.smithi203:> test -z $(ls -A /var/lib/ceph) 2024-09-17T08:09:21.987 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T08:09:22.062 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T08:09:22.075 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-17T08:09:22.075 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-17T08:09:22.075 DEBUG:teuthology.orchestra.run.smithi047:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T08:09:22.076 DEBUG:teuthology.orchestra.run.smithi193:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T08:09:22.076 DEBUG:teuthology.orchestra.run.smithi203:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T08:09:22.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:09:22.093 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-09-17T08:09:22.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:09:22.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:09:22.095 DEBUG:teuthology.orchestra.run.smithi203:> uname -r 2024-09-17T08:09:22.095 DEBUG:teuthology.orchestra.run.smithi193:> uname -r 2024-09-17T08:09:22.152 INFO:teuthology.orchestra.run.smithi047.stdout:5.14.0-503.el9.x86_64 2024-09-17T08:09:22.152 INFO:teuthology.task.kernel:Running kernel on smithi047: 5.14.0-503.el9.x86_64 2024-09-17T08:09:22.152 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum install -y kernel 2024-09-17T08:09:22.153 INFO:teuthology.orchestra.run.smithi193.stdout:5.14.0-503.el9.x86_64 2024-09-17T08:09:22.154 INFO:teuthology.task.kernel:Running kernel on smithi193: 5.14.0-503.el9.x86_64 2024-09-17T08:09:22.154 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum install -y kernel 2024-09-17T08:09:22.155 INFO:teuthology.orchestra.run.smithi203.stdout:5.14.0-503.el9.x86_64 2024-09-17T08:09:22.155 INFO:teuthology.task.kernel:Running kernel on smithi203: 5.14.0-503.el9.x86_64 2024-09-17T08:09:22.155 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum install -y kernel 2024-09-17T08:09:24.482 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.3 MB 00:00 2024-09-17T08:09:24.593 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.3 MB 00:00 2024-09-17T08:09:26.364 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.3 MB 00:02 2024-09-17T08:09:27.058 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - AppStream 14 MB/s | 20 MB 00:01 2024-09-17T08:09:29.760 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - AppStream 5.0 MB/s | 20 MB 00:04 2024-09-17T08:09:31.914 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 4.6 MB/s | 20 MB 00:04 2024-09-17T08:09:33.550 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.5 MB 00:02 2024-09-17T08:09:34.135 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-17T08:09:35.451 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 19 kB 00:00 2024-09-17T08:09:35.844 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 19 kB 00:00 2024-09-17T08:09:36.465 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-17T08:09:36.705 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-17T08:09:36.832 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-17T08:09:37.974 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 19 kB 00:00 2024-09-17T08:09:39.078 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-17T08:09:42.177 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-17T08:09:42.237 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-17T08:09:44.075 INFO:teuthology.orchestra.run.smithi193.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T08:09:44.075 INFO:teuthology.orchestra.run.smithi193.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T08:09:44.102 INFO:teuthology.orchestra.run.smithi203.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T08:09:44.102 INFO:teuthology.orchestra.run.smithi203.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T08:09:44.175 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-17T08:09:44.185 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-17T08:09:44.185 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-09-17T08:09:44.185 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-17T08:09:44.185 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-17T08:09:44.185 INFO:teuthology.orchestra.run.smithi193.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T08:09:44.185 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-09-17T08:09:44.186 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T08:09:44.186 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T08:09:44.186 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T08:09:44.186 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-17T08:09:44.186 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-17T08:09:44.186 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-17T08:09:44.186 INFO:teuthology.orchestra.run.smithi193.stdout:Install 4 Packages 2024-09-17T08:09:44.186 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-17T08:09:44.187 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 79 M 2024-09-17T08:09:44.187 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 126 M 2024-09-17T08:09:44.187 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-17T08:09:44.192 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-09-17T08:09:44.204 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-09-17T08:09:44.205 INFO:teuthology.orchestra.run.smithi203.stdout: Package Architecture Version Repository Size 2024-09-17T08:09:44.205 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-09-17T08:09:44.205 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-09-17T08:09:44.205 INFO:teuthology.orchestra.run.smithi203.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T08:09:44.205 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-09-17T08:09:44.205 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T08:09:44.205 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T08:09:44.205 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T08:09:44.205 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-17T08:09:44.205 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-09-17T08:09:44.205 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-09-17T08:09:44.205 INFO:teuthology.orchestra.run.smithi203.stdout:Install 4 Packages 2024-09-17T08:09:44.206 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-17T08:09:44.206 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 79 M 2024-09-17T08:09:44.206 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 126 M 2024-09-17T08:09:44.206 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-09-17T08:09:44.329 INFO:teuthology.orchestra.run.smithi193.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 304 kB/s | 36 kB 00:00 2024-09-17T08:09:44.414 INFO:teuthology.orchestra.run.smithi203.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 335 kB/s | 36 kB 00:00 2024-09-17T08:09:44.717 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-09-17T08:09:46.665 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T08:09:46.665 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T08:09:46.751 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-17T08:09:46.769 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-17T08:09:46.769 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-09-17T08:09:46.769 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-17T08:09:46.769 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-17T08:09:46.770 INFO:teuthology.orchestra.run.smithi047.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T08:09:46.770 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-09-17T08:09:46.770 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T08:09:46.770 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T08:09:46.770 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T08:09:46.770 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:09:46.770 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-17T08:09:46.770 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-17T08:09:46.770 INFO:teuthology.orchestra.run.smithi047.stdout:Install 4 Packages 2024-09-17T08:09:46.770 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:09:46.771 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 79 M 2024-09-17T08:09:46.771 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 126 M 2024-09-17T08:09:46.771 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-17T08:09:47.225 INFO:teuthology.orchestra.run.smithi203.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 5.4 MB/s | 16 MB 00:02 2024-09-17T08:09:47.385 INFO:teuthology.orchestra.run.smithi047.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 64 kB/s | 36 kB 00:00 2024-09-17T08:09:48.219 INFO:teuthology.orchestra.run.smithi047.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-17T08:09:49.044 INFO:teuthology.orchestra.run.smithi047.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 16 MB/s | 35 MB 00:02 2024-09-17T08:09:49.744 INFO:teuthology.orchestra.run.smithi047.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 12 MB/s | 29 MB 00:02 2024-09-17T08:09:49.744 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:09:49.745 INFO:teuthology.orchestra.run.smithi047.stdout:Total 26 MB/s | 79 MB 00:02 2024-09-17T08:09:50.459 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-17T08:09:50.505 INFO:teuthology.orchestra.run.smithi203.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 4.7 MB/s | 29 MB 00:06 2024-09-17T08:09:50.834 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-17T08:09:50.834 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-17T08:09:50.864 INFO:teuthology.orchestra.run.smithi203.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 5.3 MB/s | 35 MB 00:06 2024-09-17T08:09:50.864 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:09:50.864 INFO:teuthology.orchestra.run.smithi203.stdout:Total 12 MB/s | 79 MB 00:06 2024-09-17T08:09:51.328 INFO:teuthology.orchestra.run.smithi193.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 2.2 MB/s | 16 MB 00:07 2024-09-17T08:09:51.553 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-09-17T08:09:51.814 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-17T08:09:51.814 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-17T08:09:51.901 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-09-17T08:09:51.901 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-09-17T08:09:52.881 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-09-17T08:09:52.882 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-09-17T08:09:54.348 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-17T08:09:55.015 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T08:09:55.113 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T08:09:55.316 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-09-17T08:09:55.956 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T08:09:56.039 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T08:09:57.460 INFO:teuthology.orchestra.run.smithi193.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 2.2 MB/s | 29 MB 00:13 2024-09-17T08:09:57.855 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T08:09:57.989 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T08:09:58.818 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T08:09:58.974 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T08:10:00.319 INFO:teuthology.orchestra.run.smithi193.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 2.1 MB/s | 35 MB 00:16 2024-09-17T08:10:00.319 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:10:00.319 INFO:teuthology.orchestra.run.smithi193.stdout:Total 4.9 MB/s | 79 MB 00:16 2024-09-17T08:10:01.005 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-17T08:10:01.344 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-17T08:10:01.349 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-17T08:10:02.295 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-17T08:10:02.295 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-17T08:10:03.117 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T08:10:03.147 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:10:03.802 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T08:10:03.858 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:10:04.740 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-17T08:10:05.369 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T08:10:05.460 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T08:10:08.197 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T08:10:08.376 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:10:08.423 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T08:10:08.694 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:10:13.243 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T08:10:13.347 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:10:18.173 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:10:47.138 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:10:47.182 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:10:47.455 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:10:47.455 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T08:10:47.455 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T08:10:47.455 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T08:10:47.456 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:10:47.497 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:10:47.775 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:10:47.776 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T08:10:47.776 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T08:10:47.776 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T08:10:48.360 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:10:48.361 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-17T08:10:48.361 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-09-17T08:10:48.361 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T08:10:48.361 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T08:10:48.361 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T08:10:48.361 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T08:10:48.361 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-17T08:10:48.361 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-09-17T08:10:48.608 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:10:48.609 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:10:48.609 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-17T08:10:48.609 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T08:10:48.609 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T08:10:48.609 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T08:10:48.609 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T08:10:48.609 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:10:48.609 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-17T08:10:48.719 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T08:10:48.805 INFO:teuthology.orchestra.run.smithi203.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T08:10:48.807 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T08:10:48.807 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T08:10:48.807 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T08:10:48.807 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T08:10:48.807 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-17T08:10:48.807 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi203.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T08:10:48.808 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T08:10:48.867 INFO:teuthology.orchestra.run.smithi203.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T08:10:48.867 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T08:10:48.867 DEBUG:teuthology.orchestra.run.smithi203:> sudo rpm -qi grub2-tools 2024-09-17T08:10:48.900 INFO:teuthology.orchestra.run.smithi203.stdout:Name : grub2-tools 2024-09-17T08:10:48.900 INFO:teuthology.orchestra.run.smithi203.stdout:Epoch : 1 2024-09-17T08:10:48.900 INFO:teuthology.orchestra.run.smithi203.stdout:Version : 2.06 2024-09-17T08:10:48.900 INFO:teuthology.orchestra.run.smithi203.stdout:Release : 82.el9 2024-09-17T08:10:48.900 INFO:teuthology.orchestra.run.smithi203.stdout:Architecture: x86_64 2024-09-17T08:10:48.900 INFO:teuthology.orchestra.run.smithi203.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T08:10:48.900 INFO:teuthology.orchestra.run.smithi203.stdout:Group : Unspecified 2024-09-17T08:10:48.900 INFO:teuthology.orchestra.run.smithi203.stdout:Size : 8274670 2024-09-17T08:10:48.901 INFO:teuthology.orchestra.run.smithi203.stdout:License : GPLv3+ 2024-09-17T08:10:48.901 INFO:teuthology.orchestra.run.smithi203.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T08:10:48.901 INFO:teuthology.orchestra.run.smithi203.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T08:10:48.901 INFO:teuthology.orchestra.run.smithi203.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T08:10:48.901 INFO:teuthology.orchestra.run.smithi203.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T08:10:48.901 INFO:teuthology.orchestra.run.smithi203.stdout:Packager : builder@centos.org 2024-09-17T08:10:48.901 INFO:teuthology.orchestra.run.smithi203.stdout:Vendor : CentOS 2024-09-17T08:10:48.901 INFO:teuthology.orchestra.run.smithi203.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T08:10:48.901 INFO:teuthology.orchestra.run.smithi203.stdout:Summary : Support tools for GRUB. 2024-09-17T08:10:48.901 INFO:teuthology.orchestra.run.smithi203.stdout:Description : 2024-09-17T08:10:48.901 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-17T08:10:48.901 INFO:teuthology.orchestra.run.smithi203.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T08:10:48.901 INFO:teuthology.orchestra.run.smithi203.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T08:10:48.902 INFO:teuthology.orchestra.run.smithi203.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T08:10:48.902 INFO:teuthology.orchestra.run.smithi203.stdout:hardware devices. 2024-09-17T08:10:48.902 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-17T08:10:48.902 INFO:teuthology.orchestra.run.smithi203.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T08:10:48.903 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T08:10:48.903 INFO:teuthology.task.kernel:Updating grub on smithi203 to boot 5.14.0-508.el9.x86_64 2024-09-17T08:10:48.903 DEBUG:teuthology.orchestra.run.smithi203:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T08:10:48.924 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T08:10:49.016 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T08:10:49.016 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T08:10:49.017 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T08:10:49.017 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T08:10:49.017 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T08:10:49.017 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-17T08:10:49.017 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi047.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T08:10:49.017 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T08:10:49.075 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T08:10:49.076 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T08:10:49.076 DEBUG:teuthology.orchestra.run.smithi047:> sudo rpm -qi grub2-tools 2024-09-17T08:10:49.110 INFO:teuthology.orchestra.run.smithi047.stdout:Name : grub2-tools 2024-09-17T08:10:49.110 INFO:teuthology.orchestra.run.smithi047.stdout:Epoch : 1 2024-09-17T08:10:49.110 INFO:teuthology.orchestra.run.smithi047.stdout:Version : 2.06 2024-09-17T08:10:49.110 INFO:teuthology.orchestra.run.smithi047.stdout:Release : 82.el9 2024-09-17T08:10:49.110 INFO:teuthology.orchestra.run.smithi047.stdout:Architecture: x86_64 2024-09-17T08:10:49.111 INFO:teuthology.orchestra.run.smithi047.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T08:10:49.111 INFO:teuthology.orchestra.run.smithi047.stdout:Group : Unspecified 2024-09-17T08:10:49.111 INFO:teuthology.orchestra.run.smithi047.stdout:Size : 8274670 2024-09-17T08:10:49.111 INFO:teuthology.orchestra.run.smithi047.stdout:License : GPLv3+ 2024-09-17T08:10:49.111 INFO:teuthology.orchestra.run.smithi047.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T08:10:49.111 INFO:teuthology.orchestra.run.smithi047.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T08:10:49.111 INFO:teuthology.orchestra.run.smithi047.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T08:10:49.111 INFO:teuthology.orchestra.run.smithi047.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T08:10:49.111 INFO:teuthology.orchestra.run.smithi047.stdout:Packager : builder@centos.org 2024-09-17T08:10:49.111 INFO:teuthology.orchestra.run.smithi047.stdout:Vendor : CentOS 2024-09-17T08:10:49.111 INFO:teuthology.orchestra.run.smithi047.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T08:10:49.111 INFO:teuthology.orchestra.run.smithi047.stdout:Summary : Support tools for GRUB. 2024-09-17T08:10:49.111 INFO:teuthology.orchestra.run.smithi047.stdout:Description : 2024-09-17T08:10:49.111 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:10:49.112 INFO:teuthology.orchestra.run.smithi047.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T08:10:49.112 INFO:teuthology.orchestra.run.smithi047.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T08:10:49.112 INFO:teuthology.orchestra.run.smithi047.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T08:10:49.112 INFO:teuthology.orchestra.run.smithi047.stdout:hardware devices. 2024-09-17T08:10:49.112 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:10:49.112 INFO:teuthology.orchestra.run.smithi047.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T08:10:49.113 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T08:10:49.113 INFO:teuthology.task.kernel:Updating grub on smithi047 to boot 5.14.0-508.el9.x86_64 2024-09-17T08:10:49.113 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T08:10:49.616 INFO:teuthology.orchestra.run.smithi203.stderr:Generating grub configuration file ... 2024-09-17T08:10:49.749 INFO:teuthology.orchestra.run.smithi047.stderr:Generating grub configuration file ... 2024-09-17T08:10:50.951 INFO:teuthology.orchestra.run.smithi203.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T08:10:50.987 INFO:teuthology.orchestra.run.smithi203.stderr:done 2024-09-17T08:10:50.990 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2024-09-17T08:10:51.005 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.nrwvYzhlnp 2024-09-17T08:10:51.005 DEBUG:teuthology.orchestra.run.smithi203:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.nrwvYzhlnp 2024-09-17T08:10:51.071 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.nrwvYzhlnp 2024-09-17T08:10:51.123 INFO:teuthology.orchestra.run.smithi047.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T08:10:51.159 INFO:teuthology.orchestra.run.smithi047.stderr:done 2024-09-17T08:10:51.161 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-09-17T08:10:51.176 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.1iTzNielYK 2024-09-17T08:10:51.176 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.1iTzNielYK 2024-09-17T08:10:51.208 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.nrwvYzhlnp is 6KB 2024-09-17T08:10:51.249 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.1iTzNielYK 2024-09-17T08:10:51.256 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.nrwvYzhlnp 2024-09-17T08:10:51.271 DEBUG:teuthology.orchestra.run.smithi203:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T08:10:51.335 INFO:teuthology.orchestra.run.smithi203.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T08:10:51.335 INFO:teuthology.orchestra.run.smithi203.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T08:10:51.335 INFO:teuthology.orchestra.run.smithi203.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T08:10:51.335 INFO:teuthology.orchestra.run.smithi203.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T08:10:51.336 DEBUG:teuthology.orchestra.run.smithi203:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T08:10:51.385 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.1iTzNielYK is 6KB 2024-09-17T08:10:51.434 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.1iTzNielYK 2024-09-17T08:10:51.448 DEBUG:teuthology.orchestra.run.smithi047:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T08:10:51.491 DEBUG:teuthology.orchestra.run.smithi203:> sudo shutdown -r now 2024-09-17T08:10:51.511 INFO:teuthology.orchestra.run.smithi047.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T08:10:51.511 INFO:teuthology.orchestra.run.smithi047.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T08:10:51.511 INFO:teuthology.orchestra.run.smithi047.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T08:10:51.511 INFO:teuthology.orchestra.run.smithi047.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T08:10:51.513 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T08:10:51.706 DEBUG:teuthology.orchestra.run.smithi047:> sudo shutdown -r now 2024-09-17T08:10:56.082 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:10:56.122 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:10:56.466 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:10:56.467 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T08:10:56.467 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T08:10:56.467 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T08:10:57.317 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T08:10:57.317 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-17T08:10:57.317 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-17T08:10:57.317 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T08:10:57.317 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T08:10:57.317 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T08:10:57.317 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T08:10:57.317 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-17T08:10:57.318 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-17T08:10:57.705 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T08:10:57.788 INFO:teuthology.orchestra.run.smithi193.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T08:10:57.789 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T08:10:57.789 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T08:10:57.789 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T08:10:57.789 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T08:10:57.789 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-17T08:10:57.789 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi193.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T08:10:57.790 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T08:10:57.848 INFO:teuthology.orchestra.run.smithi193.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T08:10:57.849 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T08:10:57.849 DEBUG:teuthology.orchestra.run.smithi193:> sudo rpm -qi grub2-tools 2024-09-17T08:10:57.880 INFO:teuthology.orchestra.run.smithi193.stdout:Name : grub2-tools 2024-09-17T08:10:57.881 INFO:teuthology.orchestra.run.smithi193.stdout:Epoch : 1 2024-09-17T08:10:57.881 INFO:teuthology.orchestra.run.smithi193.stdout:Version : 2.06 2024-09-17T08:10:57.881 INFO:teuthology.orchestra.run.smithi193.stdout:Release : 82.el9 2024-09-17T08:10:57.881 INFO:teuthology.orchestra.run.smithi193.stdout:Architecture: x86_64 2024-09-17T08:10:57.881 INFO:teuthology.orchestra.run.smithi193.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T08:10:57.881 INFO:teuthology.orchestra.run.smithi193.stdout:Group : Unspecified 2024-09-17T08:10:57.881 INFO:teuthology.orchestra.run.smithi193.stdout:Size : 8274670 2024-09-17T08:10:57.881 INFO:teuthology.orchestra.run.smithi193.stdout:License : GPLv3+ 2024-09-17T08:10:57.881 INFO:teuthology.orchestra.run.smithi193.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T08:10:57.881 INFO:teuthology.orchestra.run.smithi193.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T08:10:57.881 INFO:teuthology.orchestra.run.smithi193.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T08:10:57.881 INFO:teuthology.orchestra.run.smithi193.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T08:10:57.881 INFO:teuthology.orchestra.run.smithi193.stdout:Packager : builder@centos.org 2024-09-17T08:10:57.881 INFO:teuthology.orchestra.run.smithi193.stdout:Vendor : CentOS 2024-09-17T08:10:57.882 INFO:teuthology.orchestra.run.smithi193.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T08:10:57.882 INFO:teuthology.orchestra.run.smithi193.stdout:Summary : Support tools for GRUB. 2024-09-17T08:10:57.882 INFO:teuthology.orchestra.run.smithi193.stdout:Description : 2024-09-17T08:10:57.882 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-17T08:10:57.882 INFO:teuthology.orchestra.run.smithi193.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T08:10:57.882 INFO:teuthology.orchestra.run.smithi193.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T08:10:57.882 INFO:teuthology.orchestra.run.smithi193.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T08:10:57.882 INFO:teuthology.orchestra.run.smithi193.stdout:hardware devices. 2024-09-17T08:10:57.882 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-17T08:10:57.882 INFO:teuthology.orchestra.run.smithi193.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T08:10:57.885 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T08:10:57.885 INFO:teuthology.task.kernel:Updating grub on smithi193 to boot 5.14.0-508.el9.x86_64 2024-09-17T08:10:57.885 DEBUG:teuthology.orchestra.run.smithi193:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T08:10:58.582 INFO:teuthology.orchestra.run.smithi193.stderr:Generating grub configuration file ... 2024-09-17T08:10:59.932 INFO:teuthology.orchestra.run.smithi193.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T08:10:59.951 INFO:teuthology.orchestra.run.smithi193.stderr:done 2024-09-17T08:10:59.954 DEBUG:teuthology.orchestra.run.smithi193:> mktemp 2024-09-17T08:10:59.969 INFO:teuthology.orchestra.run.smithi193.stdout:/tmp/tmp.47J0i0ipXM 2024-09-17T08:10:59.969 DEBUG:teuthology.orchestra.run.smithi193:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.47J0i0ipXM 2024-09-17T08:11:00.037 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 0666 /tmp/tmp.47J0i0ipXM 2024-09-17T08:11:00.190 DEBUG:teuthology.orchestra.remote:smithi193:/tmp/tmp.47J0i0ipXM is 6KB 2024-09-17T08:11:00.238 DEBUG:teuthology.orchestra.run.smithi193:> rm -fr /tmp/tmp.47J0i0ipXM 2024-09-17T08:11:00.253 DEBUG:teuthology.orchestra.run.smithi193:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T08:11:00.316 INFO:teuthology.orchestra.run.smithi193.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T08:11:00.316 INFO:teuthology.orchestra.run.smithi193.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T08:11:00.316 INFO:teuthology.orchestra.run.smithi193.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T08:11:00.316 INFO:teuthology.orchestra.run.smithi193.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T08:11:00.318 DEBUG:teuthology.orchestra.run.smithi193:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T08:11:00.473 DEBUG:teuthology.orchestra.run.smithi193:> sudo shutdown -r now 2024-09-17T08:11:21.511 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-17T08:11:21.512 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-09-17T08:11:21.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:11:21.710 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-17T08:11:21.710 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-17T08:11:21.711 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:11:30.485 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-17T08:11:30.485 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-17T08:11:30.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:11:39.965 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.203 2024-09-17T08:11:40.221 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.47 2024-09-17T08:11:48.925 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.193 2024-09-17T08:11:48.965 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-09-17T08:11:48.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:11:49.222 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-17T08:11:49.222 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:11:52.029 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.203 2024-09-17T08:11:57.929 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-17T08:11:57.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:12:00.989 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.193 2024-09-17T08:12:04.033 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-09-17T08:12:04.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:12:12.997 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-17T08:12:12.998 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:12:22.465 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.203 2024-09-17T08:12:37.479 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-09-17T08:12:37.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:12:37.864 DEBUG:teuthology.orchestra.run.smithi203:> true 2024-09-17T08:12:38.285 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-09-17T08:12:38.285 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-508.el9.x86_64"... 2024-09-17T08:12:38.285 DEBUG:teuthology.orchestra.run.smithi203:> uname -r 2024-09-17T08:12:38.301 INFO:teuthology.orchestra.run.smithi203.stdout:5.14.0-508.el9.x86_64 2024-09-17T08:12:38.301 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T08:12:38.301 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T08:12:38.301 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T08:12:39.302 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-17T08:12:39.302 DEBUG:teuthology.orchestra.run.smithi203:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T08:12:39.409 INFO:teuthology.orchestra.run.smithi203.stdout:ttyS1 2024-09-17T08:12:39.437 DEBUG:teuthology.parallel:result is None 2024-09-17T08:12:44.894 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-09-17T08:12:45.323 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-17T08:12:45.323 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-17T08:12:45.324 DEBUG:teuthology.orchestra.run.smithi193:> uname -r 2024-09-17T08:12:45.379 INFO:teuthology.orchestra.run.smithi193.stdout:5.14.0-508.el9.x86_64 2024-09-17T08:12:45.379 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T08:12:45.379 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T08:12:45.379 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T08:12:46.380 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-17T08:12:46.380 DEBUG:teuthology.orchestra.run.smithi193:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T08:12:46.449 INFO:teuthology.orchestra.run.smithi193.stdout:ttyS1 2024-09-17T08:12:46.466 DEBUG:teuthology.parallel:result is None 2024-09-17T08:12:49.224 DEBUG:teuthology.orchestra.remote:timed out 2024-09-17T08:13:01.225 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-17T08:13:01.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:13:01.557 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-09-17T08:13:01.910 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-17T08:13:01.910 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-17T08:13:01.910 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-09-17T08:13:01.926 INFO:teuthology.orchestra.run.smithi047.stdout:5.14.0-508.el9.x86_64 2024-09-17T08:13:01.926 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T08:13:01.926 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T08:13:01.926 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T08:13:02.927 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-17T08:13:02.927 DEBUG:teuthology.orchestra.run.smithi047:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T08:13:03.020 INFO:teuthology.orchestra.run.smithi047.stdout:ttyS1 2024-09-17T08:13:03.043 DEBUG:teuthology.parallel:result is None 2024-09-17T08:13:03.043 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T08:13:03.050 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T08:13:03.051 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T08:13:03.053 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T08:13:03.055 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T08:13:03.076 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T08:13:03.134 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T08:13:03.213 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T08:13:03.213 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T08:13:03.216 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T08:13:03.218 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T08:13:03.252 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T08:13:03.300 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T08:13:03.300 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T08:13:03.303 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T08:13:03.305 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T08:13:03.335 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T08:13:03.337 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T08:13:03.341 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T08:13:03.346 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T08:13:03.349 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T08:13:03.353 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T08:13:03.355 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T08:13:03.393 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T08:13:03.396 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T08:13:03.398 DEBUG:teuthology.orchestra.run.smithi193:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T08:13:03.401 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T08:13:03.429 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T08:13:03.452 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T08:13:03.452 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T08:13:03.467 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T08:13:03.470 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T08:13:03.487 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T08:13:03.567 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T08:13:03.613 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T08:13:03.696 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T08:13:03.738 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-17T08:13:03.738 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T08:13:03.804 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T08:13:03.849 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T08:13:03.896 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T08:13:03.979 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T08:13:04.021 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-17T08:13:04.021 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T08:13:04.088 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T08:13:04.131 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T08:13:04.196 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T08:13:04.243 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T08:13:04.286 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-17T08:13:04.286 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T08:13:04.355 DEBUG:teuthology.orchestra.run.smithi047:> sudo service rsyslog restart 2024-09-17T08:13:04.358 DEBUG:teuthology.orchestra.run.smithi193:> sudo service rsyslog restart 2024-09-17T08:13:04.360 DEBUG:teuthology.orchestra.run.smithi203:> sudo service rsyslog restart 2024-09-17T08:13:04.400 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T08:13:04.400 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T08:13:04.447 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T08:13:04.827 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T08:13:04.836 INFO:teuthology.task.internal:Starting timer... 2024-09-17T08:13:04.836 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T08:13:04.894 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T08:13:05.079 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-17T08:13:05.079 DEBUG:teuthology.orchestra.run.smithi047:> sudo service auditd rotate 2024-09-17T08:13:05.128 INFO:teuthology.orchestra.run.smithi047.stdout:Rotating logs: 2024-09-17T08:13:05.130 DEBUG:teuthology.orchestra.run.smithi193:> sudo service auditd rotate 2024-09-17T08:13:05.175 INFO:teuthology.orchestra.run.smithi193.stdout:Rotating logs: 2024-09-17T08:13:05.177 DEBUG:teuthology.orchestra.run.smithi203:> sudo service auditd rotate 2024-09-17T08:13:05.288 INFO:teuthology.orchestra.run.smithi203.stdout:Rotating logs: 2024-09-17T08:13:05.290 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T08:13:05.290 DEBUG:teuthology.orchestra.run.smithi047:> /usr/sbin/getenforce 2024-09-17T08:13:05.312 INFO:teuthology.orchestra.run.smithi047.stdout:Permissive 2024-09-17T08:13:05.312 DEBUG:teuthology.orchestra.run.smithi193:> /usr/sbin/getenforce 2024-09-17T08:13:05.335 INFO:teuthology.orchestra.run.smithi193.stdout:Permissive 2024-09-17T08:13:05.335 DEBUG:teuthology.orchestra.run.smithi203:> /usr/sbin/getenforce 2024-09-17T08:13:05.354 INFO:teuthology.orchestra.run.smithi203.stdout:Permissive 2024-09-17T08:13:05.354 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi047.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi193.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi203.front.sepia.ceph.com': 'permissive'} 2024-09-17T08:13:05.354 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T08:13:05.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:05.385 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T08:13:05.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:05.410 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T08:13:05.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:05.437 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T08:13:05.437 DEBUG:teuthology.orchestra.run.smithi047:> sudo /usr/sbin/setenforce permissive 2024-09-17T08:13:05.471 DEBUG:teuthology.orchestra.run.smithi193:> sudo /usr/sbin/setenforce permissive 2024-09-17T08:13:05.503 DEBUG:teuthology.orchestra.run.smithi203:> sudo /usr/sbin/setenforce permissive 2024-09-17T08:13:05.536 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T08:13:05.545 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T08:13:05.556 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T08:13:05.557 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi047.front.sepia.ceph.com,smithi193.front.sepia.ceph.com,smithi203.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T08:16:52.123 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi047.front.sepia.ceph.com'), Remote(name='ubuntu@smithi193.front.sepia.ceph.com'), Remote(name='ubuntu@smithi203.front.sepia.ceph.com')] 2024-09-17T08:16:52.125 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-17T08:16:52.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:16:52.206 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-09-17T08:16:52.287 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-17T08:16:52.287 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-17T08:16:52.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:16:52.363 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-09-17T08:16:52.440 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-17T08:16:52.441 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-09-17T08:16:52.441 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:16:52.515 DEBUG:teuthology.orchestra.run.smithi203:> true 2024-09-17T08:16:52.596 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-09-17T08:16:52.596 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T08:16:52.606 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T08:16:52.606 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T08:16:52.607 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T08:16:52.610 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T08:16:52.610 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T08:16:52.612 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T08:16:52.612 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T08:16:52.639 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T08:16:52.642 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T08:16:52.654 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T08:16:52.659 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T08:16:52.664 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T08:16:52.678 INFO:teuthology.orchestra.run.smithi193.stderr:sudo: ntpd: command not found 2024-09-17T08:16:52.678 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T08:16:52.687 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: ntpd: command not found 2024-09-17T08:16:52.689 INFO:teuthology.orchestra.run.smithi193.stdout:506 Cannot talk to daemon 2024-09-17T08:16:52.700 INFO:teuthology.orchestra.run.smithi047.stdout:506 Cannot talk to daemon 2024-09-17T08:16:52.702 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: ntpd: command not found 2024-09-17T08:16:52.703 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T08:16:52.714 INFO:teuthology.orchestra.run.smithi203.stdout:506 Cannot talk to daemon 2024-09-17T08:16:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T08:16:52.716 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T08:16:52.728 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T08:16:52.731 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T08:16:52.742 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T08:16:52.779 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-09-17T08:16:52.782 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T08:16:52.782 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-09-17T08:16:52.782 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:16:52.782 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:16:52.782 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:16:52.782 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:16:52.787 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-09-17T08:16:52.792 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T08:16:52.792 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-09-17T08:16:52.792 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:16:52.792 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:16:52.792 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:16:52.792 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:16:52.797 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-09-17T08:16:52.801 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T08:16:52.801 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-09-17T08:16:52.801 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:16:52.801 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:16:52.801 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:16:52.801 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:16:52.802 INFO:teuthology.run_tasks:Running task pexec... 2024-09-17T08:16:52.811 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-17T08:16:52.812 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi047.front.sepia.ceph.com 2024-09-17T08:16:52.812 DEBUG:teuthology.orchestra.run.smithi047:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-17T08:16:52.812 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi193.front.sepia.ceph.com 2024-09-17T08:16:52.812 DEBUG:teuthology.orchestra.run.smithi193:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-17T08:16:52.813 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi203.front.sepia.ceph.com 2024-09-17T08:16:52.813 DEBUG:teuthology.orchestra.run.smithi203:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-17T08:16:53.170 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-17T08:16:53.187 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-17T08:16:53.187 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-09-17T08:16:53.187 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-17T08:16:53.187 INFO:teuthology.orchestra.run.smithi193.stdout:Removing: 2024-09-17T08:16:53.187 INFO:teuthology.orchestra.run.smithi193.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-17T08:16:53.187 INFO:teuthology.orchestra.run.smithi193.stdout:Removing unused dependencies: 2024-09-17T08:16:53.187 INFO:teuthology.orchestra.run.smithi193.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-17T08:16:53.188 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-17T08:16:53.188 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-17T08:16:53.188 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-17T08:16:53.188 INFO:teuthology.orchestra.run.smithi193.stdout:Remove 2 Packages 2024-09-17T08:16:53.188 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-17T08:16:53.188 INFO:teuthology.orchestra.run.smithi193.stdout:Freed space: 5.5 M 2024-09-17T08:16:53.188 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-17T08:16:53.191 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-17T08:16:53.192 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-17T08:16:53.193 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-09-17T08:16:53.201 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-09-17T08:16:53.202 INFO:teuthology.orchestra.run.smithi203.stdout: Package Architecture Version Repository Size 2024-09-17T08:16:53.202 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-09-17T08:16:53.202 INFO:teuthology.orchestra.run.smithi203.stdout:Removing: 2024-09-17T08:16:53.202 INFO:teuthology.orchestra.run.smithi203.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-17T08:16:53.203 INFO:teuthology.orchestra.run.smithi203.stdout:Removing unused dependencies: 2024-09-17T08:16:53.203 INFO:teuthology.orchestra.run.smithi203.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-17T08:16:53.203 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-17T08:16:53.203 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-09-17T08:16:53.203 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-09-17T08:16:53.203 INFO:teuthology.orchestra.run.smithi203.stdout:Remove 2 Packages 2024-09-17T08:16:53.204 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-17T08:16:53.204 INFO:teuthology.orchestra.run.smithi203.stdout:Freed space: 5.5 M 2024-09-17T08:16:53.204 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-09-17T08:16:53.205 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-17T08:16:53.206 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-09-17T08:16:53.206 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-09-17T08:16:53.213 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-17T08:16:53.213 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-09-17T08:16:53.214 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-17T08:16:53.214 INFO:teuthology.orchestra.run.smithi047.stdout:Removing: 2024-09-17T08:16:53.214 INFO:teuthology.orchestra.run.smithi047.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-17T08:16:53.214 INFO:teuthology.orchestra.run.smithi047.stdout:Removing unused dependencies: 2024-09-17T08:16:53.214 INFO:teuthology.orchestra.run.smithi047.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-17T08:16:53.214 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:16:53.214 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-17T08:16:53.214 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-17T08:16:53.214 INFO:teuthology.orchestra.run.smithi047.stdout:Remove 2 Packages 2024-09-17T08:16:53.214 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:16:53.214 INFO:teuthology.orchestra.run.smithi047.stdout:Freed space: 5.5 M 2024-09-17T08:16:53.214 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-17T08:16:53.219 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-17T08:16:53.219 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-17T08:16:53.257 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-17T08:16:53.258 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-17T08:16:53.262 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-09-17T08:16:53.262 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-09-17T08:16:53.283 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-17T08:16:53.284 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-17T08:16:53.404 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-17T08:16:53.408 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-09-17T08:16:53.428 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-17T08:16:53.487 INFO:teuthology.orchestra.run.smithi193.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-17T08:16:53.488 INFO:teuthology.orchestra.run.smithi203.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-17T08:16:53.505 INFO:teuthology.orchestra.run.smithi047.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-17T08:16:53.541 INFO:teuthology.orchestra.run.smithi193.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T08:16:53.552 INFO:teuthology.orchestra.run.smithi203.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T08:16:53.571 INFO:teuthology.orchestra.run.smithi047.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T08:16:54.049 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T08:16:54.049 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-17T08:16:54.066 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T08:16:54.067 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-17T08:16:54.087 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T08:16:54.087 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-17T08:16:54.304 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-17T08:16:54.304 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-17T08:16:54.304 INFO:teuthology.orchestra.run.smithi203.stdout:Removed: 2024-09-17T08:16:54.304 INFO:teuthology.orchestra.run.smithi203.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T08:16:54.304 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-17T08:16:54.305 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-09-17T08:16:54.320 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-17T08:16:54.320 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-17T08:16:54.320 INFO:teuthology.orchestra.run.smithi193.stdout:Removed: 2024-09-17T08:16:54.320 INFO:teuthology.orchestra.run.smithi193.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T08:16:54.320 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-17T08:16:54.320 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-17T08:16:54.399 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-17T08:16:54.401 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:16:54.401 INFO:teuthology.orchestra.run.smithi047.stdout:Removed: 2024-09-17T08:16:54.401 INFO:teuthology.orchestra.run.smithi047.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T08:16:54.401 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:16:54.401 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-17T08:16:55.135 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:00:50 ago on Tue 17 Sep 2024 08:16:04 AM UTC. 2024-09-17T08:16:55.205 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:51 ago on Tue 17 Sep 2024 08:16:04 AM UTC. 2024-09-17T08:16:55.205 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:48 ago on Tue 17 Sep 2024 08:16:06 AM UTC. 2024-09-17T08:16:55.272 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-09-17T08:16:55.273 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-09-17T08:16:55.273 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2024-09-17T08:16:55.274 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-09-17T08:16:55.274 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-09-17T08:16:55.274 INFO:teuthology.orchestra.run.smithi203.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-17T08:16:55.274 INFO:teuthology.orchestra.run.smithi203.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-17T08:16:55.274 INFO:teuthology.orchestra.run.smithi203.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-17T08:16:55.274 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-09-17T08:16:55.274 INFO:teuthology.orchestra.run.smithi203.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-17T08:16:55.274 INFO:teuthology.orchestra.run.smithi203.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-17T08:16:55.274 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-17T08:16:55.274 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-17T08:16:55.274 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-17T08:16:55.274 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-09-17T08:16:55.274 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-09-17T08:16:55.274 INFO:teuthology.orchestra.run.smithi203.stdout:Install 7 Packages 2024-09-17T08:16:55.274 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-17T08:16:55.289 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 5.2 M 2024-09-17T08:16:55.289 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 19 M 2024-09-17T08:16:55.289 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-09-17T08:16:55.341 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-17T08:16:55.342 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-17T08:16:55.342 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-09-17T08:16:55.342 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-17T08:16:55.342 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-17T08:16:55.342 INFO:teuthology.orchestra.run.smithi193.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-17T08:16:55.342 INFO:teuthology.orchestra.run.smithi193.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-17T08:16:55.342 INFO:teuthology.orchestra.run.smithi193.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-17T08:16:55.342 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-09-17T08:16:55.342 INFO:teuthology.orchestra.run.smithi193.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-17T08:16:55.342 INFO:teuthology.orchestra.run.smithi193.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-17T08:16:55.342 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-17T08:16:55.343 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-17T08:16:55.343 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-17T08:16:55.343 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-17T08:16:55.343 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-17T08:16:55.343 INFO:teuthology.orchestra.run.smithi193.stdout:Install 7 Packages 2024-09-17T08:16:55.343 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-17T08:16:55.350 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-17T08:16:55.351 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-17T08:16:55.351 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-09-17T08:16:55.351 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-17T08:16:55.351 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-17T08:16:55.351 INFO:teuthology.orchestra.run.smithi047.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-17T08:16:55.351 INFO:teuthology.orchestra.run.smithi047.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-17T08:16:55.352 INFO:teuthology.orchestra.run.smithi047.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-17T08:16:55.352 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-09-17T08:16:55.352 INFO:teuthology.orchestra.run.smithi047.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-17T08:16:55.352 INFO:teuthology.orchestra.run.smithi047.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-17T08:16:55.352 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-17T08:16:55.352 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-17T08:16:55.352 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:16:55.352 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-17T08:16:55.352 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-17T08:16:55.352 INFO:teuthology.orchestra.run.smithi047.stdout:Install 7 Packages 2024-09-17T08:16:55.352 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:16:55.358 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 5.2 M 2024-09-17T08:16:55.358 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 19 M 2024-09-17T08:16:55.358 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-17T08:16:55.372 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 5.2 M 2024-09-17T08:16:55.372 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 19 M 2024-09-17T08:16:55.372 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-17T08:16:55.675 INFO:teuthology.orchestra.run.smithi193.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 472 kB/s | 44 kB 00:00 2024-09-17T08:16:55.699 INFO:teuthology.orchestra.run.smithi193.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 887 kB/s | 106 kB 00:00 2024-09-17T08:16:55.733 INFO:teuthology.orchestra.run.smithi193.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 1.2 MB/s | 72 kB 00:00 2024-09-17T08:16:55.758 INFO:teuthology.orchestra.run.smithi193.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.4 MB/s | 84 kB 00:00 2024-09-17T08:16:55.825 INFO:teuthology.orchestra.run.smithi193.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 3.6 MB/s | 906 kB 00:00 2024-09-17T08:16:55.901 INFO:teuthology.orchestra.run.smithi193.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 4.9 MB/s | 837 kB 00:00 2024-09-17T08:16:56.043 INFO:teuthology.orchestra.run.smithi203.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 119 kB/s | 44 kB 00:00 2024-09-17T08:16:56.143 INFO:teuthology.orchestra.run.smithi203.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 224 kB/s | 106 kB 00:00 2024-09-17T08:16:56.208 INFO:teuthology.orchestra.run.smithi047.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 75 kB/s | 44 kB 00:00 2024-09-17T08:16:56.244 INFO:teuthology.orchestra.run.smithi203.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 358 kB/s | 72 kB 00:00 2024-09-17T08:16:56.292 INFO:teuthology.orchestra.run.smithi047.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 157 kB/s | 106 kB 00:00 2024-09-17T08:16:56.344 INFO:teuthology.orchestra.run.smithi203.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 420 kB/s | 84 kB 00:00 2024-09-17T08:16:56.525 INFO:teuthology.orchestra.run.smithi047.stdout:(3/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 995 kB/s | 906 kB 00:00 2024-09-17T08:16:56.619 INFO:teuthology.orchestra.run.smithi203.stdout:(5/7): runc-1.1.13-4.el9.x86_64.rpm 12 MB/s | 3.2 MB 00:00 2024-09-17T08:16:56.653 INFO:teuthology.orchestra.run.smithi203.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 2.0 MB/s | 837 kB 00:00 2024-09-17T08:16:56.678 INFO:teuthology.orchestra.run.smithi203.stdout:(7/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 898 kB/s | 906 kB 00:01 2024-09-17T08:16:56.679 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:16:56.679 INFO:teuthology.orchestra.run.smithi203.stdout:Total 3.7 MB/s | 5.2 MB 00:01 2024-09-17T08:16:56.789 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-09-17T08:16:56.801 INFO:teuthology.orchestra.run.smithi193.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 3.0 MB/s | 3.2 MB 00:01 2024-09-17T08:16:56.801 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:16:56.801 INFO:teuthology.orchestra.run.smithi193.stdout:Total 3.6 MB/s | 5.2 MB 00:01 2024-09-17T08:16:56.804 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-09-17T08:16:56.804 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-09-17T08:16:56.859 INFO:teuthology.orchestra.run.smithi047.stdout:(4/7): python3-configshell-1.1.30-1.el9.noarch. 110 kB/s | 72 kB 00:00 2024-09-17T08:16:56.920 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-17T08:16:56.935 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-17T08:16:56.936 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-17T08:16:56.959 INFO:teuthology.orchestra.run.smithi047.stdout:(5/7): python3-kmod-0.9-32.el9.x86_64.rpm 126 kB/s | 84 kB 00:00 2024-09-17T08:16:57.132 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-09-17T08:16:57.132 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-09-17T08:16:57.271 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-17T08:16:57.271 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-17T08:16:57.417 INFO:teuthology.orchestra.run.smithi047.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 949 kB/s | 837 kB 00:00 2024-09-17T08:16:57.606 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-09-17T08:16:57.698 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-17T08:16:57.735 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-17T08:16:57.742 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-17T08:16:57.777 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-17T08:16:57.836 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-17T08:16:57.879 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-17T08:16:57.908 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-17T08:16:57.995 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-17T08:16:58.034 INFO:teuthology.orchestra.run.smithi047.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 2.7 MB/s | 3.2 MB 00:01 2024-09-17T08:16:58.034 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:16:58.035 INFO:teuthology.orchestra.run.smithi047.stdout:Total 1.9 MB/s | 5.2 MB 00:02 2024-09-17T08:16:58.035 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T08:16:58.115 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-17T08:16:58.156 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-17T08:16:58.156 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T08:16:58.172 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-17T08:16:58.173 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-17T08:16:58.469 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T08:16:58.504 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T08:16:58.506 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-17T08:16:58.506 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-17T08:16:58.602 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T08:16:58.613 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T08:16:58.639 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T08:16:58.675 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T08:16:58.753 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T08:16:58.822 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T08:16:58.991 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-17T08:16:59.080 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-17T08:16:59.114 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-17T08:16:59.142 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-17T08:16:59.362 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-17T08:16:59.398 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T08:16:59.664 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T08:16:59.664 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-17T08:16:59.664 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-17T08:16:59.664 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-17T08:16:59.664 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-17T08:16:59.664 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-17T08:16:59.664 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-17T08:16:59.822 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T08:16:59.823 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-17T08:16:59.823 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-17T08:16:59.823 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-17T08:16:59.823 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-17T08:16:59.823 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-17T08:16:59.823 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-17T08:16:59.886 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T08:16:59.929 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T08:17:00.040 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T08:17:00.045 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T08:17:00.045 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-17T08:17:00.045 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-09-17T08:17:00.045 INFO:teuthology.orchestra.run.smithi203.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T08:17:00.045 INFO:teuthology.orchestra.run.smithi203.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-17T08:17:00.046 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-17T08:17:00.046 INFO:teuthology.orchestra.run.smithi203.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-17T08:17:00.046 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-17T08:17:00.046 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-09-17T08:17:00.107 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T08:17:00.196 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T08:17:00.196 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-17T08:17:00.197 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-17T08:17:00.197 INFO:teuthology.orchestra.run.smithi193.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T08:17:00.197 INFO:teuthology.orchestra.run.smithi193.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-17T08:17:00.197 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-17T08:17:00.197 INFO:teuthology.orchestra.run.smithi193.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-17T08:17:00.197 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-17T08:17:00.197 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-17T08:17:00.444 DEBUG:teuthology.parallel:result is None 2024-09-17T08:17:00.589 DEBUG:teuthology.parallel:result is None 2024-09-17T08:17:01.082 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T08:17:01.082 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-17T08:17:01.082 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-17T08:17:01.082 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-17T08:17:01.082 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-17T08:17:01.082 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-17T08:17:01.083 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-17T08:17:01.473 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T08:17:01.473 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:17:01.473 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-17T08:17:01.473 INFO:teuthology.orchestra.run.smithi047.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T08:17:01.473 INFO:teuthology.orchestra.run.smithi047.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-17T08:17:01.473 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-17T08:17:01.473 INFO:teuthology.orchestra.run.smithi047.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-17T08:17:01.473 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:17:01.473 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-17T08:17:01.858 DEBUG:teuthology.parallel:result is None 2024-09-17T08:17:01.858 INFO:teuthology.run_tasks:Running task nvme_loop... 2024-09-17T08:17:01.867 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices... 2024-09-17T08:17:01.867 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-17T08:17:01.868 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T08:17:01.884 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T08:17:01.884 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_1 2024-09-17T08:17:01.940 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T08:17:01.940 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:17:01.940 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 964 Links: 1 2024-09-17T08:17:01.940 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:17:01.940 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:17:01.941 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-17 08:17:00.334432089 +0000 2024-09-17T08:17:01.941 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-17 08:16:59.993439678 +0000 2024-09-17T08:17:01.941 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-17 08:16:59.993439678 +0000 2024-09-17T08:17:01.941 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-17 08:16:59.993439678 +0000 2024-09-17T08:17:01.941 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T08:17:02.004 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-17T08:17:02.004 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-17T08:17:02.004 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000110923 s, 4.6 MB/s 2024-09-17T08:17:02.006 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T08:17:02.063 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_2 2024-09-17T08:17:02.119 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T08:17:02.119 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:17:02.119 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 969 Links: 1 2024-09-17T08:17:02.119 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:17:02.120 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:17:02.120 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-17 08:17:00.334432089 +0000 2024-09-17T08:17:02.120 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-17 08:16:59.993439678 +0000 2024-09-17T08:17:02.120 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-17 08:16:59.993439678 +0000 2024-09-17T08:17:02.120 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-17 08:16:59.993439678 +0000 2024-09-17T08:17:02.120 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T08:17:02.183 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-17T08:17:02.184 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-17T08:17:02.184 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000104547 s, 4.9 MB/s 2024-09-17T08:17:02.185 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T08:17:02.242 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_3 2024-09-17T08:17:02.298 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T08:17:02.298 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:17:02.299 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 966 Links: 1 2024-09-17T08:17:02.299 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:17:02.299 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:17:02.299 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-17 08:17:00.334432089 +0000 2024-09-17T08:17:02.299 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-17 08:16:59.993439678 +0000 2024-09-17T08:17:02.299 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-17 08:16:59.993439678 +0000 2024-09-17T08:17:02.299 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-17 08:16:59.993439678 +0000 2024-09-17T08:17:02.299 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T08:17:02.363 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-17T08:17:02.363 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-17T08:17:02.363 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000102686 s, 5.0 MB/s 2024-09-17T08:17:02.364 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T08:17:02.421 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_4 2024-09-17T08:17:02.477 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T08:17:02.477 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:17:02.477 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 961 Links: 1 2024-09-17T08:17:02.477 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:17:02.477 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:17:02.477 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-17 08:17:00.335432067 +0000 2024-09-17T08:17:02.478 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-17 08:16:59.993439678 +0000 2024-09-17T08:17:02.478 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-17 08:16:59.993439678 +0000 2024-09-17T08:17:02.478 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-17 08:16:59.993439678 +0000 2024-09-17T08:17:02.478 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T08:17:02.541 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-17T08:17:02.542 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-17T08:17:02.542 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 9.8836e-05 s, 5.2 MB/s 2024-09-17T08:17:02.543 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T08:17:02.600 DEBUG:teuthology.orchestra.run.smithi047:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-17T08:17:02.766 INFO:teuthology.orchestra.run.smithi047.stdout:loop 2024-09-17T08:17:02.768 INFO:tasks.nvme_loop:Connecting nvme_loop smithi047:/dev/vg_nvme/lv_1... 2024-09-17T08:17:02.768 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-17T08:17:02.804 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-09-17T08:17:02.837 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/vg_nvme/lv_11 2024-09-17T08:17:02.880 INFO:teuthology.orchestra.run.smithi047.stdout:connecting to device: nvme1 2024-09-17T08:17:02.882 INFO:tasks.nvme_loop:Connecting nvme_loop smithi047:/dev/vg_nvme/lv_2... 2024-09-17T08:17:02.882 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-17T08:17:02.957 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-09-17T08:17:02.990 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/vg_nvme/lv_21 2024-09-17T08:17:03.022 INFO:teuthology.orchestra.run.smithi047.stdout:connecting to device: nvme2 2024-09-17T08:17:03.024 INFO:tasks.nvme_loop:Connecting nvme_loop smithi047:/dev/vg_nvme/lv_3... 2024-09-17T08:17:03.024 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-17T08:17:03.100 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-09-17T08:17:03.131 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/vg_nvme/lv_31 2024-09-17T08:17:03.162 INFO:teuthology.orchestra.run.smithi047.stdout:connecting to device: nvme3 2024-09-17T08:17:03.165 INFO:tasks.nvme_loop:Connecting nvme_loop smithi047:/dev/vg_nvme/lv_4... 2024-09-17T08:17:03.165 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-17T08:17:03.240 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-09-17T08:17:03.300 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/vg_nvme/lv_41 2024-09-17T08:17:03.333 INFO:teuthology.orchestra.run.smithi047.stdout:connecting to device: nvme4 2024-09-17T08:17:03.335 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-17T08:17:03.335 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T08:17:03.391 DEBUG:teuthology.orchestra.run.smithi047:> sudo nvme list -o json 2024-09-17T08:17:03.458 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-17T08:17:03.458 INFO:teuthology.orchestra.run.smithi047.stdout: "Devices":[ 2024-09-17T08:17:03.459 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-17T08:17:03.459 INFO:teuthology.orchestra.run.smithi047.stdout: "NameSpace":1, 2024-09-17T08:17:03.459 INFO:teuthology.orchestra.run.smithi047.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-17T08:17:03.459 INFO:teuthology.orchestra.run.smithi047.stdout: "GenericPath":"/dev/ng4n1", 2024-09-17T08:17:03.459 INFO:teuthology.orchestra.run.smithi047.stdout: "Firmware":"5.14.0-5", 2024-09-17T08:17:03.459 INFO:teuthology.orchestra.run.smithi047.stdout: "ModelNumber":"Linux", 2024-09-17T08:17:03.459 INFO:teuthology.orchestra.run.smithi047.stdout: "SerialNumber":"6103c9bfef571206e4d1", 2024-09-17T08:17:03.459 INFO:teuthology.orchestra.run.smithi047.stdout: "UsedBytes":95995035648, 2024-09-17T08:17:03.459 INFO:teuthology.orchestra.run.smithi047.stdout: "MaximumLBA":187490304, 2024-09-17T08:17:03.459 INFO:teuthology.orchestra.run.smithi047.stdout: "PhysicalSize":95995035648, 2024-09-17T08:17:03.459 INFO:teuthology.orchestra.run.smithi047.stdout: "SectorSize":512 2024-09-17T08:17:03.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-17T08:17:03.459 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-17T08:17:03.460 INFO:teuthology.orchestra.run.smithi047.stdout: "NameSpace":1, 2024-09-17T08:17:03.460 INFO:teuthology.orchestra.run.smithi047.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-17T08:17:03.460 INFO:teuthology.orchestra.run.smithi047.stdout: "GenericPath":"/dev/ng3n1", 2024-09-17T08:17:03.460 INFO:teuthology.orchestra.run.smithi047.stdout: "Firmware":"5.14.0-5", 2024-09-17T08:17:03.460 INFO:teuthology.orchestra.run.smithi047.stdout: "ModelNumber":"Linux", 2024-09-17T08:17:03.460 INFO:teuthology.orchestra.run.smithi047.stdout: "SerialNumber":"072b873147b212ae6904", 2024-09-17T08:17:03.460 INFO:teuthology.orchestra.run.smithi047.stdout: "UsedBytes":95995035648, 2024-09-17T08:17:03.460 INFO:teuthology.orchestra.run.smithi047.stdout: "MaximumLBA":187490304, 2024-09-17T08:17:03.460 INFO:teuthology.orchestra.run.smithi047.stdout: "PhysicalSize":95995035648, 2024-09-17T08:17:03.460 INFO:teuthology.orchestra.run.smithi047.stdout: "SectorSize":512 2024-09-17T08:17:03.460 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-17T08:17:03.460 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-17T08:17:03.460 INFO:teuthology.orchestra.run.smithi047.stdout: "NameSpace":1, 2024-09-17T08:17:03.461 INFO:teuthology.orchestra.run.smithi047.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-17T08:17:03.461 INFO:teuthology.orchestra.run.smithi047.stdout: "GenericPath":"/dev/ng2n1", 2024-09-17T08:17:03.461 INFO:teuthology.orchestra.run.smithi047.stdout: "Firmware":"5.14.0-5", 2024-09-17T08:17:03.461 INFO:teuthology.orchestra.run.smithi047.stdout: "ModelNumber":"Linux", 2024-09-17T08:17:03.461 INFO:teuthology.orchestra.run.smithi047.stdout: "SerialNumber":"a2d34d393cc9b2254f8e", 2024-09-17T08:17:03.461 INFO:teuthology.orchestra.run.smithi047.stdout: "UsedBytes":95995035648, 2024-09-17T08:17:03.461 INFO:teuthology.orchestra.run.smithi047.stdout: "MaximumLBA":187490304, 2024-09-17T08:17:03.461 INFO:teuthology.orchestra.run.smithi047.stdout: "PhysicalSize":95995035648, 2024-09-17T08:17:03.461 INFO:teuthology.orchestra.run.smithi047.stdout: "SectorSize":512 2024-09-17T08:17:03.461 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-17T08:17:03.461 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-17T08:17:03.462 INFO:teuthology.orchestra.run.smithi047.stdout: "NameSpace":1, 2024-09-17T08:17:03.462 INFO:teuthology.orchestra.run.smithi047.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-17T08:17:03.462 INFO:teuthology.orchestra.run.smithi047.stdout: "GenericPath":"/dev/ng1n1", 2024-09-17T08:17:03.462 INFO:teuthology.orchestra.run.smithi047.stdout: "Firmware":"5.14.0-5", 2024-09-17T08:17:03.462 INFO:teuthology.orchestra.run.smithi047.stdout: "ModelNumber":"Linux", 2024-09-17T08:17:03.462 INFO:teuthology.orchestra.run.smithi047.stdout: "SerialNumber":"053f7dfeea999baddc61", 2024-09-17T08:17:03.462 INFO:teuthology.orchestra.run.smithi047.stdout: "UsedBytes":95995035648, 2024-09-17T08:17:03.462 INFO:teuthology.orchestra.run.smithi047.stdout: "MaximumLBA":187490304, 2024-09-17T08:17:03.463 INFO:teuthology.orchestra.run.smithi047.stdout: "PhysicalSize":95995035648, 2024-09-17T08:17:03.463 INFO:teuthology.orchestra.run.smithi047.stdout: "SectorSize":512 2024-09-17T08:17:03.463 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-17T08:17:03.463 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-17T08:17:03.463 INFO:teuthology.orchestra.run.smithi047.stdout: "NameSpace":1, 2024-09-17T08:17:03.463 INFO:teuthology.orchestra.run.smithi047.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-17T08:17:03.463 INFO:teuthology.orchestra.run.smithi047.stdout: "GenericPath":"/dev/ng0n1", 2024-09-17T08:17:03.463 INFO:teuthology.orchestra.run.smithi047.stdout: "Firmware":"E2010325", 2024-09-17T08:17:03.463 INFO:teuthology.orchestra.run.smithi047.stdout: "ModelNumber":"INTEL SSDPED1D480GA", 2024-09-17T08:17:03.463 INFO:teuthology.orchestra.run.smithi047.stdout: "SerialNumber":"PHMB75130022480DGN", 2024-09-17T08:17:03.463 INFO:teuthology.orchestra.run.smithi047.stdout: "UsedBytes":480103981056, 2024-09-17T08:17:03.463 INFO:teuthology.orchestra.run.smithi047.stdout: "MaximumLBA":937703088, 2024-09-17T08:17:03.463 INFO:teuthology.orchestra.run.smithi047.stdout: "PhysicalSize":480103981056, 2024-09-17T08:17:03.463 INFO:teuthology.orchestra.run.smithi047.stdout: "SectorSize":512 2024-09-17T08:17:03.464 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-17T08:17:03.464 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-17T08:17:03.464 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-17T08:17:03.465 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-17T08:17:03.465 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-17T08:17:03.465 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/scratch_devs 2024-09-17T08:17:03.530 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-17T08:17:03.531 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T08:17:03.551 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T08:17:03.551 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_1 2024-09-17T08:17:03.610 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T08:17:03.610 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:17:03.610 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 976 Links: 1 2024-09-17T08:17:03.610 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:17:03.610 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:17:03.610 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-17 08:16:59.047652657 +0000 2024-09-17T08:17:03.610 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-17 08:16:58.712657883 +0000 2024-09-17T08:17:03.610 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-17 08:16:58.712657883 +0000 2024-09-17T08:17:03.610 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-17 08:16:58.712657883 +0000 2024-09-17T08:17:03.611 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T08:17:03.677 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-17T08:17:03.677 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-17T08:17:03.678 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000205458 s, 2.5 MB/s 2024-09-17T08:17:03.679 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T08:17:03.738 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_2 2024-09-17T08:17:03.795 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T08:17:03.795 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:17:03.795 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 969 Links: 1 2024-09-17T08:17:03.795 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:17:03.795 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:17:03.796 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-17 08:16:59.047652657 +0000 2024-09-17T08:17:03.796 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-17 08:16:58.711657899 +0000 2024-09-17T08:17:03.796 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-17 08:16:58.711657899 +0000 2024-09-17T08:17:03.796 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-17 08:16:58.711657899 +0000 2024-09-17T08:17:03.796 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T08:17:03.862 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-17T08:17:03.862 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-17T08:17:03.862 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000262542 s, 2.0 MB/s 2024-09-17T08:17:03.863 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T08:17:03.923 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_3 2024-09-17T08:17:03.980 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T08:17:03.981 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:17:03.981 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 984 Links: 1 2024-09-17T08:17:03.981 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:17:03.981 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:17:03.981 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-17 08:16:59.048652642 +0000 2024-09-17T08:17:03.981 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-17 08:16:58.712657883 +0000 2024-09-17T08:17:03.981 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-17 08:16:58.712657883 +0000 2024-09-17T08:17:03.981 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-17 08:16:58.712657883 +0000 2024-09-17T08:17:03.981 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T08:17:04.048 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-17T08:17:04.048 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-17T08:17:04.048 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000234896 s, 2.2 MB/s 2024-09-17T08:17:04.049 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T08:17:04.109 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_4 2024-09-17T08:17:04.167 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T08:17:04.167 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:17:04.167 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 965 Links: 1 2024-09-17T08:17:04.167 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:17:04.167 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:17:04.167 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-17 08:16:59.048652642 +0000 2024-09-17T08:17:04.167 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-17 08:16:58.711657899 +0000 2024-09-17T08:17:04.167 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-17 08:16:58.711657899 +0000 2024-09-17T08:17:04.167 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-17 08:16:58.711657899 +0000 2024-09-17T08:17:04.168 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T08:17:04.233 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-17T08:17:04.233 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-17T08:17:04.233 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000166047 s, 3.1 MB/s 2024-09-17T08:17:04.234 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T08:17:04.292 DEBUG:teuthology.orchestra.run.smithi193:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-17T08:17:04.430 INFO:teuthology.orchestra.run.smithi193.stdout:loop 2024-09-17T08:17:04.432 INFO:tasks.nvme_loop:Connecting nvme_loop smithi193:/dev/vg_nvme/lv_1... 2024-09-17T08:17:04.432 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-17T08:17:04.465 INFO:teuthology.orchestra.run.smithi193.stdout:1 2024-09-17T08:17:04.498 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/vg_nvme/lv_11 2024-09-17T08:17:04.536 INFO:teuthology.orchestra.run.smithi193.stdout:connecting to device: nvme1 2024-09-17T08:17:04.538 INFO:tasks.nvme_loop:Connecting nvme_loop smithi193:/dev/vg_nvme/lv_2... 2024-09-17T08:17:04.538 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-17T08:17:04.612 INFO:teuthology.orchestra.run.smithi193.stdout:1 2024-09-17T08:17:04.655 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/vg_nvme/lv_21 2024-09-17T08:17:04.690 INFO:teuthology.orchestra.run.smithi193.stdout:connecting to device: nvme2 2024-09-17T08:17:04.692 INFO:tasks.nvme_loop:Connecting nvme_loop smithi193:/dev/vg_nvme/lv_3... 2024-09-17T08:17:04.693 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-17T08:17:04.764 INFO:teuthology.orchestra.run.smithi193.stdout:1 2024-09-17T08:17:04.794 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/vg_nvme/lv_31 2024-09-17T08:17:04.830 INFO:teuthology.orchestra.run.smithi193.stdout:connecting to device: nvme3 2024-09-17T08:17:04.832 INFO:tasks.nvme_loop:Connecting nvme_loop smithi193:/dev/vg_nvme/lv_4... 2024-09-17T08:17:04.832 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-17T08:17:04.907 INFO:teuthology.orchestra.run.smithi193.stdout:1 2024-09-17T08:17:04.938 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/vg_nvme/lv_41 2024-09-17T08:17:04.968 INFO:teuthology.orchestra.run.smithi193.stdout:connecting to device: nvme4 2024-09-17T08:17:04.970 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-17T08:17:04.970 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T08:17:05.026 DEBUG:teuthology.orchestra.run.smithi193:> sudo nvme list -o json 2024-09-17T08:17:05.093 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-17T08:17:05.093 INFO:teuthology.orchestra.run.smithi193.stdout: "Devices":[ 2024-09-17T08:17:05.093 INFO:teuthology.orchestra.run.smithi193.stdout: { 2024-09-17T08:17:05.093 INFO:teuthology.orchestra.run.smithi193.stdout: "NameSpace":1, 2024-09-17T08:17:05.093 INFO:teuthology.orchestra.run.smithi193.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-17T08:17:05.093 INFO:teuthology.orchestra.run.smithi193.stdout: "GenericPath":"/dev/ng4n1", 2024-09-17T08:17:05.093 INFO:teuthology.orchestra.run.smithi193.stdout: "Firmware":"5.14.0-5", 2024-09-17T08:17:05.093 INFO:teuthology.orchestra.run.smithi193.stdout: "ModelNumber":"Linux", 2024-09-17T08:17:05.093 INFO:teuthology.orchestra.run.smithi193.stdout: "SerialNumber":"75491bf8ac535f5fd462", 2024-09-17T08:17:05.093 INFO:teuthology.orchestra.run.smithi193.stdout: "UsedBytes":95995035648, 2024-09-17T08:17:05.094 INFO:teuthology.orchestra.run.smithi193.stdout: "MaximumLBA":187490304, 2024-09-17T08:17:05.094 INFO:teuthology.orchestra.run.smithi193.stdout: "PhysicalSize":95995035648, 2024-09-17T08:17:05.094 INFO:teuthology.orchestra.run.smithi193.stdout: "SectorSize":512 2024-09-17T08:17:05.094 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-17T08:17:05.094 INFO:teuthology.orchestra.run.smithi193.stdout: { 2024-09-17T08:17:05.094 INFO:teuthology.orchestra.run.smithi193.stdout: "NameSpace":1, 2024-09-17T08:17:05.094 INFO:teuthology.orchestra.run.smithi193.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-17T08:17:05.094 INFO:teuthology.orchestra.run.smithi193.stdout: "GenericPath":"/dev/ng3n1", 2024-09-17T08:17:05.094 INFO:teuthology.orchestra.run.smithi193.stdout: "Firmware":"5.14.0-5", 2024-09-17T08:17:05.094 INFO:teuthology.orchestra.run.smithi193.stdout: "ModelNumber":"Linux", 2024-09-17T08:17:05.094 INFO:teuthology.orchestra.run.smithi193.stdout: "SerialNumber":"8c2bd542fdcb63d32d20", 2024-09-17T08:17:05.094 INFO:teuthology.orchestra.run.smithi193.stdout: "UsedBytes":95995035648, 2024-09-17T08:17:05.094 INFO:teuthology.orchestra.run.smithi193.stdout: "MaximumLBA":187490304, 2024-09-17T08:17:05.095 INFO:teuthology.orchestra.run.smithi193.stdout: "PhysicalSize":95995035648, 2024-09-17T08:17:05.095 INFO:teuthology.orchestra.run.smithi193.stdout: "SectorSize":512 2024-09-17T08:17:05.095 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-17T08:17:05.095 INFO:teuthology.orchestra.run.smithi193.stdout: { 2024-09-17T08:17:05.095 INFO:teuthology.orchestra.run.smithi193.stdout: "NameSpace":1, 2024-09-17T08:17:05.095 INFO:teuthology.orchestra.run.smithi193.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-17T08:17:05.095 INFO:teuthology.orchestra.run.smithi193.stdout: "GenericPath":"/dev/ng2n1", 2024-09-17T08:17:05.095 INFO:teuthology.orchestra.run.smithi193.stdout: "Firmware":"5.14.0-5", 2024-09-17T08:17:05.095 INFO:teuthology.orchestra.run.smithi193.stdout: "ModelNumber":"Linux", 2024-09-17T08:17:05.095 INFO:teuthology.orchestra.run.smithi193.stdout: "SerialNumber":"71bb9e14d4d0843f10b1", 2024-09-17T08:17:05.095 INFO:teuthology.orchestra.run.smithi193.stdout: "UsedBytes":95995035648, 2024-09-17T08:17:05.095 INFO:teuthology.orchestra.run.smithi193.stdout: "MaximumLBA":187490304, 2024-09-17T08:17:05.095 INFO:teuthology.orchestra.run.smithi193.stdout: "PhysicalSize":95995035648, 2024-09-17T08:17:05.095 INFO:teuthology.orchestra.run.smithi193.stdout: "SectorSize":512 2024-09-17T08:17:05.096 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-17T08:17:05.096 INFO:teuthology.orchestra.run.smithi193.stdout: { 2024-09-17T08:17:05.096 INFO:teuthology.orchestra.run.smithi193.stdout: "NameSpace":1, 2024-09-17T08:17:05.096 INFO:teuthology.orchestra.run.smithi193.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-17T08:17:05.096 INFO:teuthology.orchestra.run.smithi193.stdout: "GenericPath":"/dev/ng1n1", 2024-09-17T08:17:05.096 INFO:teuthology.orchestra.run.smithi193.stdout: "Firmware":"5.14.0-5", 2024-09-17T08:17:05.096 INFO:teuthology.orchestra.run.smithi193.stdout: "ModelNumber":"Linux", 2024-09-17T08:17:05.096 INFO:teuthology.orchestra.run.smithi193.stdout: "SerialNumber":"ce79d4faf185e7f09943", 2024-09-17T08:17:05.096 INFO:teuthology.orchestra.run.smithi193.stdout: "UsedBytes":95995035648, 2024-09-17T08:17:05.096 INFO:teuthology.orchestra.run.smithi193.stdout: "MaximumLBA":187490304, 2024-09-17T08:17:05.096 INFO:teuthology.orchestra.run.smithi193.stdout: "PhysicalSize":95995035648, 2024-09-17T08:17:05.096 INFO:teuthology.orchestra.run.smithi193.stdout: "SectorSize":512 2024-09-17T08:17:05.096 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-17T08:17:05.096 INFO:teuthology.orchestra.run.smithi193.stdout: { 2024-09-17T08:17:05.097 INFO:teuthology.orchestra.run.smithi193.stdout: "NameSpace":1, 2024-09-17T08:17:05.097 INFO:teuthology.orchestra.run.smithi193.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-17T08:17:05.097 INFO:teuthology.orchestra.run.smithi193.stdout: "GenericPath":"/dev/ng0n1", 2024-09-17T08:17:05.097 INFO:teuthology.orchestra.run.smithi193.stdout: "Firmware":"8DV101H0", 2024-09-17T08:17:05.097 INFO:teuthology.orchestra.run.smithi193.stdout: "ModelNumber":"INTEL SSDPEDMD400G4", 2024-09-17T08:17:05.097 INFO:teuthology.orchestra.run.smithi193.stdout: "SerialNumber":"PHFT620500HB400BGN", 2024-09-17T08:17:05.097 INFO:teuthology.orchestra.run.smithi193.stdout: "UsedBytes":400088457216, 2024-09-17T08:17:05.097 INFO:teuthology.orchestra.run.smithi193.stdout: "MaximumLBA":781422768, 2024-09-17T08:17:05.097 INFO:teuthology.orchestra.run.smithi193.stdout: "PhysicalSize":400088457216, 2024-09-17T08:17:05.097 INFO:teuthology.orchestra.run.smithi193.stdout: "SectorSize":512 2024-09-17T08:17:05.097 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-17T08:17:05.097 INFO:teuthology.orchestra.run.smithi193.stdout: ] 2024-09-17T08:17:05.097 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-17T08:17:05.098 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-17T08:17:05.098 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-17T08:17:05.099 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/scratch_devs 2024-09-17T08:17:05.163 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-17T08:17:05.163 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T08:17:05.183 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T08:17:05.183 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_1 2024-09-17T08:17:05.241 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T08:17:05.241 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:17:05.241 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 970 Links: 1 2024-09-17T08:17:05.241 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:17:05.241 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:17:05.241 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-09-17 08:16:58.880408510 +0000 2024-09-17T08:17:05.242 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-09-17 08:16:58.579481290 +0000 2024-09-17T08:17:05.242 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-09-17 08:16:58.579481290 +0000 2024-09-17T08:17:05.242 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-09-17 08:16:58.579481290 +0000 2024-09-17T08:17:05.242 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T08:17:05.309 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-09-17T08:17:05.309 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-09-17T08:17:05.309 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000223035 s, 2.3 MB/s 2024-09-17T08:17:05.311 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T08:17:05.369 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_2 2024-09-17T08:17:05.425 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T08:17:05.425 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:17:05.425 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 982 Links: 1 2024-09-17T08:17:05.425 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:17:05.426 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:17:05.426 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-09-17 08:16:58.880408510 +0000 2024-09-17T08:17:05.426 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-09-17 08:16:58.580480618 +0000 2024-09-17T08:17:05.426 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-09-17 08:16:58.580480618 +0000 2024-09-17T08:17:05.426 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-09-17 08:16:58.580480618 +0000 2024-09-17T08:17:05.426 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T08:17:05.491 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-09-17T08:17:05.491 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-09-17T08:17:05.491 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000283977 s, 1.8 MB/s 2024-09-17T08:17:05.492 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T08:17:05.549 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_3 2024-09-17T08:17:05.605 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T08:17:05.606 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:17:05.606 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 980 Links: 1 2024-09-17T08:17:05.606 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:17:05.606 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:17:05.606 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-09-17 08:16:58.881408486 +0000 2024-09-17T08:17:05.606 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-09-17 08:16:58.580480618 +0000 2024-09-17T08:17:05.606 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-09-17 08:16:58.580480618 +0000 2024-09-17T08:17:05.606 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-09-17 08:16:58.580480618 +0000 2024-09-17T08:17:05.606 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T08:17:05.669 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-09-17T08:17:05.669 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-09-17T08:17:05.669 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000216474 s, 2.4 MB/s 2024-09-17T08:17:05.670 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T08:17:05.727 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_4 2024-09-17T08:17:05.783 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T08:17:05.783 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:17:05.783 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 967 Links: 1 2024-09-17T08:17:05.784 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:17:05.784 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:17:05.784 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-09-17 08:16:58.881408486 +0000 2024-09-17T08:17:05.784 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-09-17 08:16:58.579481290 +0000 2024-09-17T08:17:05.784 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-09-17 08:16:58.579481290 +0000 2024-09-17T08:17:05.784 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-09-17 08:16:58.579481290 +0000 2024-09-17T08:17:05.784 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T08:17:05.847 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-09-17T08:17:05.847 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-09-17T08:17:05.847 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.00021282 s, 2.4 MB/s 2024-09-17T08:17:05.848 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T08:17:05.905 DEBUG:teuthology.orchestra.run.smithi203:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-17T08:17:06.045 INFO:teuthology.orchestra.run.smithi203.stdout:loop 2024-09-17T08:17:06.046 INFO:tasks.nvme_loop:Connecting nvme_loop smithi203:/dev/vg_nvme/lv_1... 2024-09-17T08:17:06.046 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-17T08:17:06.079 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-09-17T08:17:06.108 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/vg_nvme/lv_11 2024-09-17T08:17:06.157 INFO:teuthology.orchestra.run.smithi203.stdout:connecting to device: nvme1 2024-09-17T08:17:06.159 INFO:tasks.nvme_loop:Connecting nvme_loop smithi203:/dev/vg_nvme/lv_2... 2024-09-17T08:17:06.160 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-17T08:17:06.237 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-09-17T08:17:06.268 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/vg_nvme/lv_21 2024-09-17T08:17:06.304 INFO:teuthology.orchestra.run.smithi203.stdout:connecting to device: nvme2 2024-09-17T08:17:06.306 INFO:tasks.nvme_loop:Connecting nvme_loop smithi203:/dev/vg_nvme/lv_3... 2024-09-17T08:17:06.306 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-17T08:17:06.382 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-09-17T08:17:06.414 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/vg_nvme/lv_31 2024-09-17T08:17:06.445 INFO:teuthology.orchestra.run.smithi203.stdout:connecting to device: nvme3 2024-09-17T08:17:06.446 INFO:tasks.nvme_loop:Connecting nvme_loop smithi203:/dev/vg_nvme/lv_4... 2024-09-17T08:17:06.446 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-17T08:17:06.519 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-09-17T08:17:06.549 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/vg_nvme/lv_41 2024-09-17T08:17:06.585 INFO:teuthology.orchestra.run.smithi203.stdout:connecting to device: nvme4 2024-09-17T08:17:06.587 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-17T08:17:06.587 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T08:17:06.642 DEBUG:teuthology.orchestra.run.smithi203:> sudo nvme list -o json 2024-09-17T08:17:06.708 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-09-17T08:17:06.709 INFO:teuthology.orchestra.run.smithi203.stdout: "Devices":[ 2024-09-17T08:17:06.709 INFO:teuthology.orchestra.run.smithi203.stdout: { 2024-09-17T08:17:06.709 INFO:teuthology.orchestra.run.smithi203.stdout: "NameSpace":1, 2024-09-17T08:17:06.709 INFO:teuthology.orchestra.run.smithi203.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-17T08:17:06.709 INFO:teuthology.orchestra.run.smithi203.stdout: "GenericPath":"/dev/ng4n1", 2024-09-17T08:17:06.709 INFO:teuthology.orchestra.run.smithi203.stdout: "Firmware":"5.14.0-5", 2024-09-17T08:17:06.709 INFO:teuthology.orchestra.run.smithi203.stdout: "ModelNumber":"Linux", 2024-09-17T08:17:06.709 INFO:teuthology.orchestra.run.smithi203.stdout: "SerialNumber":"3dc0b248e4c662526a12", 2024-09-17T08:17:06.709 INFO:teuthology.orchestra.run.smithi203.stdout: "UsedBytes":95995035648, 2024-09-17T08:17:06.709 INFO:teuthology.orchestra.run.smithi203.stdout: "MaximumLBA":187490304, 2024-09-17T08:17:06.709 INFO:teuthology.orchestra.run.smithi203.stdout: "PhysicalSize":95995035648, 2024-09-17T08:17:06.709 INFO:teuthology.orchestra.run.smithi203.stdout: "SectorSize":512 2024-09-17T08:17:06.709 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-17T08:17:06.709 INFO:teuthology.orchestra.run.smithi203.stdout: { 2024-09-17T08:17:06.710 INFO:teuthology.orchestra.run.smithi203.stdout: "NameSpace":1, 2024-09-17T08:17:06.710 INFO:teuthology.orchestra.run.smithi203.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-17T08:17:06.710 INFO:teuthology.orchestra.run.smithi203.stdout: "GenericPath":"/dev/ng3n1", 2024-09-17T08:17:06.710 INFO:teuthology.orchestra.run.smithi203.stdout: "Firmware":"5.14.0-5", 2024-09-17T08:17:06.710 INFO:teuthology.orchestra.run.smithi203.stdout: "ModelNumber":"Linux", 2024-09-17T08:17:06.710 INFO:teuthology.orchestra.run.smithi203.stdout: "SerialNumber":"038c12331e5bd189c77d", 2024-09-17T08:17:06.710 INFO:teuthology.orchestra.run.smithi203.stdout: "UsedBytes":95995035648, 2024-09-17T08:17:06.710 INFO:teuthology.orchestra.run.smithi203.stdout: "MaximumLBA":187490304, 2024-09-17T08:17:06.710 INFO:teuthology.orchestra.run.smithi203.stdout: "PhysicalSize":95995035648, 2024-09-17T08:17:06.710 INFO:teuthology.orchestra.run.smithi203.stdout: "SectorSize":512 2024-09-17T08:17:06.710 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-17T08:17:06.710 INFO:teuthology.orchestra.run.smithi203.stdout: { 2024-09-17T08:17:06.710 INFO:teuthology.orchestra.run.smithi203.stdout: "NameSpace":1, 2024-09-17T08:17:06.710 INFO:teuthology.orchestra.run.smithi203.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-17T08:17:06.711 INFO:teuthology.orchestra.run.smithi203.stdout: "GenericPath":"/dev/ng2n1", 2024-09-17T08:17:06.711 INFO:teuthology.orchestra.run.smithi203.stdout: "Firmware":"5.14.0-5", 2024-09-17T08:17:06.711 INFO:teuthology.orchestra.run.smithi203.stdout: "ModelNumber":"Linux", 2024-09-17T08:17:06.711 INFO:teuthology.orchestra.run.smithi203.stdout: "SerialNumber":"406c44bc223c3a803c86", 2024-09-17T08:17:06.711 INFO:teuthology.orchestra.run.smithi203.stdout: "UsedBytes":95995035648, 2024-09-17T08:17:06.711 INFO:teuthology.orchestra.run.smithi203.stdout: "MaximumLBA":187490304, 2024-09-17T08:17:06.711 INFO:teuthology.orchestra.run.smithi203.stdout: "PhysicalSize":95995035648, 2024-09-17T08:17:06.711 INFO:teuthology.orchestra.run.smithi203.stdout: "SectorSize":512 2024-09-17T08:17:06.711 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-17T08:17:06.711 INFO:teuthology.orchestra.run.smithi203.stdout: { 2024-09-17T08:17:06.711 INFO:teuthology.orchestra.run.smithi203.stdout: "NameSpace":1, 2024-09-17T08:17:06.711 INFO:teuthology.orchestra.run.smithi203.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-17T08:17:06.711 INFO:teuthology.orchestra.run.smithi203.stdout: "GenericPath":"/dev/ng1n1", 2024-09-17T08:17:06.711 INFO:teuthology.orchestra.run.smithi203.stdout: "Firmware":"5.14.0-5", 2024-09-17T08:17:06.711 INFO:teuthology.orchestra.run.smithi203.stdout: "ModelNumber":"Linux", 2024-09-17T08:17:06.711 INFO:teuthology.orchestra.run.smithi203.stdout: "SerialNumber":"265e35466071d7b72f61", 2024-09-17T08:17:06.712 INFO:teuthology.orchestra.run.smithi203.stdout: "UsedBytes":95995035648, 2024-09-17T08:17:06.712 INFO:teuthology.orchestra.run.smithi203.stdout: "MaximumLBA":187490304, 2024-09-17T08:17:06.712 INFO:teuthology.orchestra.run.smithi203.stdout: "PhysicalSize":95995035648, 2024-09-17T08:17:06.712 INFO:teuthology.orchestra.run.smithi203.stdout: "SectorSize":512 2024-09-17T08:17:06.712 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-17T08:17:06.712 INFO:teuthology.orchestra.run.smithi203.stdout: { 2024-09-17T08:17:06.712 INFO:teuthology.orchestra.run.smithi203.stdout: "NameSpace":1, 2024-09-17T08:17:06.712 INFO:teuthology.orchestra.run.smithi203.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-17T08:17:06.712 INFO:teuthology.orchestra.run.smithi203.stdout: "GenericPath":"/dev/ng0n1", 2024-09-17T08:17:06.712 INFO:teuthology.orchestra.run.smithi203.stdout: "Firmware":"8DV101H0", 2024-09-17T08:17:06.712 INFO:teuthology.orchestra.run.smithi203.stdout: "ModelNumber":"INTEL SSDPEDMD400G4", 2024-09-17T08:17:06.712 INFO:teuthology.orchestra.run.smithi203.stdout: "SerialNumber":"CVFT623300GK400BGN", 2024-09-17T08:17:06.712 INFO:teuthology.orchestra.run.smithi203.stdout: "UsedBytes":400088457216, 2024-09-17T08:17:06.712 INFO:teuthology.orchestra.run.smithi203.stdout: "MaximumLBA":781422768, 2024-09-17T08:17:06.712 INFO:teuthology.orchestra.run.smithi203.stdout: "PhysicalSize":400088457216, 2024-09-17T08:17:06.713 INFO:teuthology.orchestra.run.smithi203.stdout: "SectorSize":512 2024-09-17T08:17:06.713 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-17T08:17:06.713 INFO:teuthology.orchestra.run.smithi203.stdout: ] 2024-09-17T08:17:06.713 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-09-17T08:17:06.714 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-17T08:17:06.714 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-17T08:17:06.714 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/scratch_devs 2024-09-17T08:17:06.778 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-17T08:17:06.867 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': False}, 'global': {'mon election default strategy': 3}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'CEPHADM_AGENT_DOWN', 'CEPHADM_FAILED_DAEMON'], 'log-only-match': ['CEPHADM_'], 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'} 2024-09-17T08:17:06.867 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:17:06.867 INFO:tasks.cephadm:Cluster fsid is 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:17:06.867 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-17T08:17:06.868 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.47', 'mon.b': '172.21.15.193', 'mon.c': '172.21.15.203'} 2024-09-17T08:17:06.868 INFO:tasks.cephadm:First mon is mon.a on smithi047 2024-09-17T08:17:06.868 INFO:tasks.cephadm:First mgr is y 2024-09-17T08:17:06.868 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-17T08:17:06.868 DEBUG:teuthology.orchestra.run.smithi047:> sudo hostname $(hostname -s) 2024-09-17T08:17:06.900 DEBUG:teuthology.orchestra.run.smithi193:> sudo hostname $(hostname -s) 2024-09-17T08:17:06.931 DEBUG:teuthology.orchestra.run.smithi203:> sudo hostname $(hostname -s) 2024-09-17T08:17:06.956 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-17T08:17:06.957 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:17:07.098 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56094/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.4-855-g19e2a2be', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.4-855.g19e2a2be'}, 'url': 'https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-16 16:29:21.739533', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'reef', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-17T08:17:07.241 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref reef, sha1 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:17:07.243 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/x86_64/flavors/default/cephadm 2024-09-17T08:17:07.243 DEBUG:teuthology.orchestra.run.smithi047:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-17T08:17:07.584 INFO:teuthology.orchestra.run.smithi047.stdout:-rw-r--r--. 1 ubuntu ubuntu 217222 Sep 17 08:17 /home/ubuntu/cephtest/cephadm 2024-09-17T08:17:07.584 DEBUG:teuthology.orchestra.run.smithi193:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-17T08:17:07.918 INFO:teuthology.orchestra.run.smithi193.stdout:-rw-r--r--. 1 ubuntu ubuntu 217222 Sep 17 08:17 /home/ubuntu/cephtest/cephadm 2024-09-17T08:17:07.918 DEBUG:teuthology.orchestra.run.smithi203:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-17T08:17:08.237 INFO:teuthology.orchestra.run.smithi203.stdout:-rw-r--r--. 1 ubuntu ubuntu 217222 Sep 17 08:17 /home/ubuntu/cephtest/cephadm 2024-09-17T08:17:08.238 DEBUG:teuthology.orchestra.run.smithi047:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-17T08:17:08.258 DEBUG:teuthology.orchestra.run.smithi193:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-17T08:17:08.278 DEBUG:teuthology.orchestra.run.smithi203:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-17T08:17:08.307 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 on all hosts... 2024-09-17T08:17:08.307 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 pull 2024-09-17T08:17:08.310 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 pull 2024-09-17T08:17:08.321 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 pull 2024-09-17T08:17:09.774 INFO:teuthology.orchestra.run.smithi047.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T08:17:09.830 INFO:teuthology.orchestra.run.smithi203.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T08:17:09.867 INFO:teuthology.orchestra.run.smithi193.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T08:17:39.625 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-17T08:17:39.626 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_version": "ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable)", 2024-09-17T08:17:39.626 INFO:teuthology.orchestra.run.smithi193.stdout: "image_id": "980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89", 2024-09-17T08:17:39.626 INFO:teuthology.orchestra.run.smithi193.stdout: "repo_digests": [ 2024-09-17T08:17:39.626 INFO:teuthology.orchestra.run.smithi193.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9" 2024-09-17T08:17:39.626 INFO:teuthology.orchestra.run.smithi193.stdout: ] 2024-09-17T08:17:39.626 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-17T08:17:45.550 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-17T08:17:45.550 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable)", 2024-09-17T08:17:45.550 INFO:teuthology.orchestra.run.smithi047.stdout: "image_id": "980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89", 2024-09-17T08:17:45.550 INFO:teuthology.orchestra.run.smithi047.stdout: "repo_digests": [ 2024-09-17T08:17:45.550 INFO:teuthology.orchestra.run.smithi047.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9" 2024-09-17T08:17:45.550 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-17T08:17:45.550 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-17T08:17:45.874 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-09-17T08:17:45.875 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_version": "ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable)", 2024-09-17T08:17:45.875 INFO:teuthology.orchestra.run.smithi203.stdout: "image_id": "980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89", 2024-09-17T08:17:45.875 INFO:teuthology.orchestra.run.smithi203.stdout: "repo_digests": [ 2024-09-17T08:17:45.875 INFO:teuthology.orchestra.run.smithi203.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9" 2024-09-17T08:17:45.875 INFO:teuthology.orchestra.run.smithi203.stdout: ] 2024-09-17T08:17:45.875 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-09-17T08:17:45.894 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph 2024-09-17T08:17:45.925 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /etc/ceph 2024-09-17T08:17:45.958 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph 2024-09-17T08:17:45.985 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 777 /etc/ceph 2024-09-17T08:17:46.013 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 777 /etc/ceph 2024-09-17T08:17:46.041 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 777 /etc/ceph 2024-09-17T08:17:46.066 INFO:tasks.cephadm:Writing seed config... 2024-09-17T08:17:46.067 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-17T08:17:46.067 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-17T08:17:46.067 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = False 2024-09-17T08:17:46.067 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-09-17T08:17:46.067 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-17T08:17:46.067 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-17T08:17:46.067 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-17T08:17:46.067 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-17T08:17:46.068 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-17T08:17:46.068 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-17T08:17:46.068 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-17T08:17:46.068 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-17T08:17:46.085 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3a66518a-74cd-11ef-bceb-c7b262605968 mon election default strategy = 3 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = False [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-17T08:17:46.086 DEBUG:teuthology.orchestra.run.smithi047:mon.a> sudo journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mon.a.service 2024-09-17T08:17:46.128 DEBUG:teuthology.orchestra.run.smithi047:mgr.y> sudo journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mgr.y.service 2024-09-17T08:17:46.170 INFO:tasks.cephadm:Bootstrapping... 2024-09-17T08:17:46.170 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 -v bootstrap --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.47 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:17:46.328 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:17:46.328 INFO:teuthology.orchestra.run.smithi047.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', '-v', 'bootstrap', '--fsid', '3a66518a-74cd-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.47', '--skip-admin-label'] 2024-09-17T08:17:46.352 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 5.2.2 2024-09-17T08:17:46.353 INFO:teuthology.orchestra.run.smithi047.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-17T08:17:46.353 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying podman|docker is present... 2024-09-17T08:17:46.375 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 5.2.2 2024-09-17T08:17:46.375 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying lvm2 is present... 2024-09-17T08:17:46.375 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying time synchronization is in place... 2024-09-17T08:17:46.382 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-17T08:17:46.382 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-17T08:17:46.389 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-17T08:17:46.389 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-09-17T08:17:46.396 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout enabled 2024-09-17T08:17:46.402 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout active 2024-09-17T08:17:46.402 INFO:teuthology.orchestra.run.smithi047.stdout:Unit chronyd.service is enabled and running 2024-09-17T08:17:46.402 INFO:teuthology.orchestra.run.smithi047.stdout:Repeating the final host check... 2024-09-17T08:17:46.425 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 5.2.2 2024-09-17T08:17:46.425 INFO:teuthology.orchestra.run.smithi047.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-17T08:17:46.425 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl is present 2024-09-17T08:17:46.425 INFO:teuthology.orchestra.run.smithi047.stdout:lvcreate is present 2024-09-17T08:17:46.432 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-17T08:17:46.432 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-17T08:17:46.438 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-17T08:17:46.438 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-09-17T08:17:46.445 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout enabled 2024-09-17T08:17:46.453 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout active 2024-09-17T08:17:46.454 INFO:teuthology.orchestra.run.smithi047.stdout:Unit chronyd.service is enabled and running 2024-09-17T08:17:46.454 INFO:teuthology.orchestra.run.smithi047.stdout:Host looks OK 2024-09-17T08:17:46.454 INFO:teuthology.orchestra.run.smithi047.stdout:Cluster fsid: 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:17:46.454 INFO:teuthology.orchestra.run.smithi047.stdout:Acquiring lock 140645270073504 on /run/cephadm/3a66518a-74cd-11ef-bceb-c7b262605968.lock 2024-09-17T08:17:46.454 INFO:teuthology.orchestra.run.smithi047.stdout:Lock 140645270073504 acquired on /run/cephadm/3a66518a-74cd-11ef-bceb-c7b262605968.lock 2024-09-17T08:17:46.454 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying IP 172.21.15.47 port 3300 ... 2024-09-17T08:17:46.454 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying IP 172.21.15.47 port 6789 ... 2024-09-17T08:17:46.454 INFO:teuthology.orchestra.run.smithi047.stdout:Base mon IP(s) is [172.21.15.47:3300, 172.21.15.47:6789], mon addrv is [v2:172.21.15.47:3300,v1:172.21.15.47:6789] 2024-09-17T08:17:46.456 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.47 metric 100 2024-09-17T08:17:46.456 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.47 metric 100 2024-09-17T08:17:46.459 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-17T08:17:46.459 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-17T08:17:46.459 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1795sec hoplimit 64 pref medium 2024-09-17T08:17:46.461 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-17T08:17:46.462 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-17T08:17:46.462 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-17T08:17:46.462 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-17T08:17:46.462 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:148e/64 scope link 2024-09-17T08:17:46.462 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-17T08:17:46.462 INFO:teuthology.orchestra.run.smithi047.stdout:Mon IP `172.21.15.47` is in CIDR network `172.21.0.0/20` 2024-09-17T08:17:46.463 INFO:teuthology.orchestra.run.smithi047.stdout:Mon IP `172.21.15.47` is in CIDR network `172.21.0.0/20` 2024-09-17T08:17:46.463 INFO:teuthology.orchestra.run.smithi047.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-17T08:17:46.463 INFO:teuthology.orchestra.run.smithi047.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-17T08:17:46.464 INFO:teuthology.orchestra.run.smithi047.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T08:17:46.895 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 2024-09-17T08:17:46.895 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T08:17:46.895 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Getting image source signatures 2024-09-17T08:17:46.895 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Copying blob sha256:d8a707c21dc06a64315160cb2d8e524fa91943a344731b2a930ee3a716bebbdd 2024-09-17T08:17:46.895 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-17T08:17:46.895 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Copying config sha256:980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 2024-09-17T08:17:46.895 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-17T08:17:47.775 INFO:teuthology.orchestra.run.smithi047.stdout:ceph: stdout ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable) 2024-09-17T08:17:47.775 INFO:teuthology.orchestra.run.smithi047.stdout:Ceph version: ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable) 2024-09-17T08:17:47.775 INFO:teuthology.orchestra.run.smithi047.stdout:Extracting ceph user uid/gid from container image... 2024-09-17T08:17:48.764 INFO:teuthology.orchestra.run.smithi047.stdout:stat: stdout 167 167 2024-09-17T08:17:48.764 INFO:teuthology.orchestra.run.smithi047.stdout:Creating initial keys... 2024-09-17T08:17:51.546 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph-authtool: stdout AQAtO+lmq6mgERAA3Bu2KPKwZzQMk87bV6bDUw== 2024-09-17T08:17:52.435 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph-authtool: stdout AQAvO+lm4nQINBAA3JtgUPGi+GSNnqp19Sv92Q== 2024-09-17T08:17:53.373 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph-authtool: stdout AQAwO+lmlsLrLxAAgbYRvoAiMua3zzLYzT1cEQ== 2024-09-17T08:17:53.374 INFO:teuthology.orchestra.run.smithi047.stdout:Creating initial monmap... 2024-09-17T08:17:54.262 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-17T08:17:54.263 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-09-17T08:17:54.263 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:17:54.263 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-17T08:17:54.263 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool for a [v2:172.21.15.47:3300,v1:172.21.15.47:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-17T08:17:54.263 INFO:teuthology.orchestra.run.smithi047.stdout:setting min_mon_release = pacific 2024-09-17T08:17:54.263 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: set fsid to 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:17:54.263 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-17T08:17:54.263 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:17:54.263 INFO:teuthology.orchestra.run.smithi047.stdout:Creating mon... 2024-09-17T08:17:55.201 INFO:teuthology.orchestra.run.smithi047.stdout:create mon.a on 2024-09-17T08:17:55.571 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-17T08:17:55.767 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3a66518a-74cd-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-3a66518a-74cd-11ef-bceb-c7b262605968.target. 2024-09-17T08:17:55.767 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3a66518a-74cd-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-3a66518a-74cd-11ef-bceb-c7b262605968.target. 2024-09-17T08:17:55.993 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mon.a 2024-09-17T08:17:55.994 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-17T08:17:56.159 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3a66518a-74cd-11ef-bceb-c7b262605968.target.wants/ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-3a66518a-74cd-11ef-bceb-c7b262605968@.service. 2024-09-17T08:17:56.493 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:17:56 smithi047 podman[25354]: 2024-09-17 08:17:56.301665084 +0000 UTC m=+0.016134985 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:17:56.625 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T08:17:56.625 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-09-17T08:17:56.632 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T08:17:56.632 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-09-17T08:17:56.632 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-09-17T08:17:56.632 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to enable service . firewalld.service is not available 2024-09-17T08:17:56.633 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mon to start... 2024-09-17T08:17:56.633 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mon... 2024-09-17T08:17:56.827 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:17:56 smithi047 podman[25354]: 2024-09-17 08:17:56.553593956 +0000 UTC m=+0.268063857 container init 9525ae7cf0252690bae2efef814ebb76e2df1f2634e2fb1b9f613d7ced82a405 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-17T08:17:56.827 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:17:56 smithi047 podman[25354]: 2024-09-17 08:17:56.558594006 +0000 UTC m=+0.273063896 container start 9525ae7cf0252690bae2efef814ebb76e2df1f2634e2fb1b9f613d7ced82a405 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-17T08:17:56.827 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:17:56 smithi047 ceph-mon[25392]: mkfs 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:17:56.827 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:17:56 smithi047 ceph-mon[25392]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-17T08:17:56.827 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:17:56 smithi047 bash[25354]: 9525ae7cf0252690bae2efef814ebb76e2df1f2634e2fb1b9f613d7ced82a405 2024-09-17T08:17:56.827 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:17:56 smithi047 systemd[1]: Started Ceph mon.a for 3a66518a-74cd-11ef-bceb-c7b262605968. 2024-09-17T08:17:57.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout cluster: 2024-09-17T08:17:57.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout id: 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:17:57.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-17T08:17:57.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-17T08:17:57.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout services: 2024-09-17T08:17:57.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.503404s) 2024-09-17T08:17:57.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-17T08:17:57.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-17T08:17:57.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-17T08:17:57.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout data: 2024-09-17T08:17:57.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-17T08:17:57.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-17T08:17:57.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-17T08:17:57.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout pgs: 2024-09-17T08:17:57.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-17T08:17:57.640 INFO:teuthology.orchestra.run.smithi047.stdout:mon is available 2024-09-17T08:17:57.640 INFO:teuthology.orchestra.run.smithi047.stdout:Assimilating anything we can from ceph.conf... 2024-09-17T08:17:57.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:17:57 smithi047 ceph-mon[25392]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-17T08:17:57.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:17:57 smithi047 ceph-mon[25392]: monmap e1: 1 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:17:57.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:17:57 smithi047 ceph-mon[25392]: fsmap 2024-09-17T08:17:57.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:17:57 smithi047 ceph-mon[25392]: osdmap e1: 0 total, 0 up, 0 in 2024-09-17T08:17:57.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:17:57 smithi047 ceph-mon[25392]: mgrmap e1: no daemons active 2024-09-17T08:17:57.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:17:57 smithi047 ceph-mon[25392]: from='client.? 172.21.15.47:0/322722632' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-17T08:17:58.682 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:17:58 smithi047 ceph-mon[25392]: from='client.? 172.21.15.47:0/3385983278' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-17T08:17:58.682 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:17:58 smithi047 ceph-mon[25392]: from='client.? 172.21.15.47:0/3385983278' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-17T08:17:58.686 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-17T08:17:58.686 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [global] 2024-09-17T08:17:58.686 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout fsid = 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:17:58.686 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.47:3300,v1:172.21.15.47:6789] 2024-09-17T08:17:58.686 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-17T08:17:58.686 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-17T08:17:58.686 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-17T08:17:58.686 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-17T08:17:58.686 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-17T08:17:58.686 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-17T08:17:58.687 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr/cephadm/use_agent = False 2024-09-17T08:17:58.687 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-17T08:17:58.687 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-17T08:17:58.687 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [osd] 2024-09-17T08:17:58.687 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-17T08:17:58.687 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-17T08:17:58.687 INFO:teuthology.orchestra.run.smithi047.stdout:Generating new minimal ceph.conf... 2024-09-17T08:17:59.599 INFO:teuthology.orchestra.run.smithi047.stdout:Restarting the monitor... 2024-09-17T08:17:59.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:17:59 smithi047 ceph-mon[25392]: from='client.? 172.21.15.47:0/3668624435' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:17:59.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:17:59 smithi047 systemd[1]: Stopping Ceph mon.a for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:18:00.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:17:59 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-a[25369]: 2024-09-17T08:17:59.876+0000 7f72550c4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:18:00.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:17:59 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-a[25369]: 2024-09-17T08:17:59.876+0000 7f72550c4640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-17T08:18:00.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:00 smithi047 podman[25712]: 2024-09-17 08:18:00.12323981 +0000 UTC m=+0.361509334 container died 9525ae7cf0252690bae2efef814ebb76e2df1f2634e2fb1b9f613d7ced82a405 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-17T08:18:00.617 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:00 smithi047 podman[25712]: 2024-09-17 08:18:00.432479705 +0000 UTC m=+0.670749298 container cleanup 9525ae7cf0252690bae2efef814ebb76e2df1f2634e2fb1b9f613d7ced82a405 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-17T08:18:00.617 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:00 smithi047 bash[25712]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-a 2024-09-17T08:18:01.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:00 smithi047 podman[25729]: 2024-09-17 08:18:00.616559392 +0000 UTC m=+0.496633784 container remove 9525ae7cf0252690bae2efef814ebb76e2df1f2634e2fb1b9f613d7ced82a405 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-a, ceph=True, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-17T08:18:01.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:00 smithi047 systemd[1]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-17T08:18:01.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:00 smithi047 systemd[1]: Stopped Ceph mon.a for 3a66518a-74cd-11ef-bceb-c7b262605968. 2024-09-17T08:18:01.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:00 smithi047 systemd[1]: Starting Ceph mon.a for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:18:01.320 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 podman[25827]: 2024-09-17 08:18:01.042241765 +0000 UTC m=+0.090863470 container create a0d16c90f0857baa4dd2177ab84ed678902f3e968a053395d6b5394c8041c0a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-17T08:18:01.320 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 podman[25827]: 2024-09-17 08:18:00.967702384 +0000 UTC m=+0.016324104 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:18:01.320 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 podman[25827]: 2024-09-17 08:18:01.244950851 +0000 UTC m=+0.293572563 container init a0d16c90f0857baa4dd2177ab84ed678902f3e968a053395d6b5394c8041c0a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-17T08:18:01.320 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 podman[25827]: 2024-09-17 08:18:01.249725846 +0000 UTC m=+0.298347545 container start a0d16c90f0857baa4dd2177ab84ed678902f3e968a053395d6b5394c8041c0a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-17T08:18:01.320 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: set uid:gid to 167:167 (ceph:ceph) 2024-09-17T08:18:01.320 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable), process ceph-mon, pid 8 2024-09-17T08:18:01.320 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: pidfile_write: ignore empty --pid-file 2024-09-17T08:18:01.320 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: load: jerasure load: lrc 2024-09-17T08:18:01.320 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: RocksDB version: 7.9.2 2024-09-17T08:18:01.321 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Git sha 0 2024-09-17T08:18:01.321 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Compile date 2024-09-16 15:15:13 2024-09-17T08:18:01.321 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: DB SUMMARY 2024-09-17T08:18:01.321 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: DB Session ID: JEUSJ37VFY5VEETFY314 2024-09-17T08:18:01.321 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: CURRENT file: CURRENT 2024-09-17T08:18:01.321 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: IDENTITY file: IDENTITY 2024-09-17T08:18:01.321 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-17T08:18:01.321 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-17T08:18:01.321 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 83736 ; 2024-09-17T08:18:01.321 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.error_if_exists: 0 2024-09-17T08:18:01.321 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.create_if_missing: 0 2024-09-17T08:18:01.321 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.paranoid_checks: 1 2024-09-17T08:18:01.321 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-17T08:18:01.322 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-17T08:18:01.322 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-17T08:18:01.322 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.env: 0x5591036ecbc0 2024-09-17T08:18:01.322 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.fs: PosixFileSystem 2024-09-17T08:18:01.322 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.info_log: 0x559104998be0 2024-09-17T08:18:01.322 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_file_opening_threads: 16 2024-09-17T08:18:01.322 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.statistics: (nil) 2024-09-17T08:18:01.322 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.use_fsync: 0 2024-09-17T08:18:01.322 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_log_file_size: 0 2024-09-17T08:18:01.322 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-17T08:18:01.322 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-17T08:18:01.322 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.keep_log_file_num: 1000 2024-09-17T08:18:01.322 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.recycle_log_file_num: 0 2024-09-17T08:18:01.323 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.allow_fallocate: 1 2024-09-17T08:18:01.323 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.allow_mmap_reads: 0 2024-09-17T08:18:01.323 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.allow_mmap_writes: 0 2024-09-17T08:18:01.323 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.use_direct_reads: 0 2024-09-17T08:18:01.323 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-17T08:18:01.323 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.create_missing_column_families: 0 2024-09-17T08:18:01.323 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.db_log_dir: 2024-09-17T08:18:01.323 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.wal_dir: 2024-09-17T08:18:01.323 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-17T08:18:01.323 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-17T08:18:01.323 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-17T08:18:01.323 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-17T08:18:01.323 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-17T08:18:01.323 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-17T08:18:01.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.advise_random_on_open: 1 2024-09-17T08:18:01.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.db_write_buffer_size: 0 2024-09-17T08:18:01.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.write_buffer_manager: 0x5591049a6320 2024-09-17T08:18:01.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-17T08:18:01.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-17T08:18:01.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-17T08:18:01.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.rate_limiter: (nil) 2024-09-17T08:18:01.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-17T08:18:01.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.wal_recovery_mode: 2 2024-09-17T08:18:01.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.enable_thread_tracking: 0 2024-09-17T08:18:01.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.enable_pipelined_write: 0 2024-09-17T08:18:01.326 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.unordered_write: 0 2024-09-17T08:18:01.326 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-17T08:18:01.326 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-17T08:18:01.326 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-17T08:18:01.326 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-17T08:18:01.326 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.row_cache: None 2024-09-17T08:18:01.326 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.wal_filter: None 2024-09-17T08:18:01.326 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-17T08:18:01.326 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.allow_ingest_behind: 0 2024-09-17T08:18:01.326 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.two_write_queues: 0 2024-09-17T08:18:01.326 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.manual_wal_flush: 0 2024-09-17T08:18:01.326 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.wal_compression: 0 2024-09-17T08:18:01.326 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.atomic_flush: 0 2024-09-17T08:18:01.327 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-17T08:18:01.327 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-17T08:18:01.327 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-17T08:18:01.327 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.log_readahead_size: 0 2024-09-17T08:18:01.327 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-17T08:18:01.327 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.best_efforts_recovery: 0 2024-09-17T08:18:01.327 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-17T08:18:01.327 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-17T08:18:01.327 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.allow_data_in_errors: 0 2024-09-17T08:18:01.327 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.db_host_id: __hostname__ 2024-09-17T08:18:01.327 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-17T08:18:01.327 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_background_jobs: 2 2024-09-17T08:18:01.327 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_background_compactions: -1 2024-09-17T08:18:01.327 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_subcompactions: 1 2024-09-17T08:18:01.328 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-17T08:18:01.328 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-17T08:18:01.328 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-17T08:18:01.328 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_total_wal_size: 0 2024-09-17T08:18:01.328 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-17T08:18:01.328 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-17T08:18:01.328 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-17T08:18:01.328 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-17T08:18:01.328 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_open_files: -1 2024-09-17T08:18:01.328 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.bytes_per_sync: 0 2024-09-17T08:18:01.328 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-17T08:18:01.328 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-17T08:18:01.328 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compaction_readahead_size: 0 2024-09-17T08:18:01.328 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_background_flushes: -1 2024-09-17T08:18:01.329 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Compression algorithms supported: 2024-09-17T08:18:01.329 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: kZSTD supported: 0 2024-09-17T08:18:01.329 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: kXpressCompression supported: 0 2024-09-17T08:18:01.329 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: kBZip2Compression supported: 0 2024-09-17T08:18:01.329 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-17T08:18:01.329 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: kLZ4Compression supported: 1 2024-09-17T08:18:01.329 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: kZlibCompression supported: 1 2024-09-17T08:18:01.329 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: kLZ4HCCompression supported: 1 2024-09-17T08:18:01.329 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: kSnappyCompression supported: 1 2024-09-17T08:18:01.329 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-17T08:18:01.329 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-17T08:18:01.329 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-17T08:18:01.329 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-17T08:18:01.329 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-17T08:18:01.330 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.merge_operator: 2024-09-17T08:18:01.330 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compaction_filter: None 2024-09-17T08:18:01.330 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compaction_filter_factory: None 2024-09-17T08:18:01.330 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.sst_partitioner_factory: None 2024-09-17T08:18:01.330 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-17T08:18:01.330 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-17T08:18:01.330 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5591049987e0) 2024-09-17T08:18:01.330 INFO:journalctl@ceph.mon.a.smithi047.stdout: cache_index_and_filter_blocks: 1 2024-09-17T08:18:01.330 INFO:journalctl@ceph.mon.a.smithi047.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-17T08:18:01.330 INFO:journalctl@ceph.mon.a.smithi047.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-17T08:18:01.330 INFO:journalctl@ceph.mon.a.smithi047.stdout: pin_top_level_index_and_filter: 1 2024-09-17T08:18:01.330 INFO:journalctl@ceph.mon.a.smithi047.stdout: index_type: 0 2024-09-17T08:18:01.330 INFO:journalctl@ceph.mon.a.smithi047.stdout: data_block_index_type: 0 2024-09-17T08:18:01.330 INFO:journalctl@ceph.mon.a.smithi047.stdout: index_shortening: 1 2024-09-17T08:18:01.331 INFO:journalctl@ceph.mon.a.smithi047.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-17T08:18:01.331 INFO:journalctl@ceph.mon.a.smithi047.stdout: checksum: 4 2024-09-17T08:18:01.331 INFO:journalctl@ceph.mon.a.smithi047.stdout: no_block_cache: 0 2024-09-17T08:18:01.331 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache: 0x55910498d350 2024-09-17T08:18:01.331 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache_name: BinnedLRUCache 2024-09-17T08:18:01.331 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache_options: 2024-09-17T08:18:01.331 INFO:journalctl@ceph.mon.a.smithi047.stdout: capacity : 536870912 2024-09-17T08:18:01.331 INFO:journalctl@ceph.mon.a.smithi047.stdout: num_shard_bits : 4 2024-09-17T08:18:01.331 INFO:journalctl@ceph.mon.a.smithi047.stdout: strict_capacity_limit : 0 2024-09-17T08:18:01.331 INFO:journalctl@ceph.mon.a.smithi047.stdout: high_pri_pool_ratio: 0.000 2024-09-17T08:18:01.331 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache_compressed: (nil) 2024-09-17T08:18:01.331 INFO:journalctl@ceph.mon.a.smithi047.stdout: persistent_cache: (nil) 2024-09-17T08:18:01.331 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_size: 4096 2024-09-17T08:18:01.331 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_size_deviation: 10 2024-09-17T08:18:01.332 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_restart_interval: 16 2024-09-17T08:18:01.332 INFO:journalctl@ceph.mon.a.smithi047.stdout: index_block_restart_interval: 1 2024-09-17T08:18:01.332 INFO:journalctl@ceph.mon.a.smithi047.stdout: metadata_block_size: 4096 2024-09-17T08:18:01.332 INFO:journalctl@ceph.mon.a.smithi047.stdout: partition_filters: 0 2024-09-17T08:18:01.332 INFO:journalctl@ceph.mon.a.smithi047.stdout: use_delta_encoding: 1 2024-09-17T08:18:01.332 INFO:journalctl@ceph.mon.a.smithi047.stdout: filter_policy: bloomfilter 2024-09-17T08:18:01.332 INFO:journalctl@ceph.mon.a.smithi047.stdout: whole_key_filtering: 1 2024-09-17T08:18:01.332 INFO:journalctl@ceph.mon.a.smithi047.stdout: verify_compression: 0 2024-09-17T08:18:01.332 INFO:journalctl@ceph.mon.a.smithi047.stdout: read_amp_bytes_per_bit: 0 2024-09-17T08:18:01.332 INFO:journalctl@ceph.mon.a.smithi047.stdout: format_version: 5 2024-09-17T08:18:01.332 INFO:journalctl@ceph.mon.a.smithi047.stdout: enable_index_compression: 1 2024-09-17T08:18:01.332 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_align: 0 2024-09-17T08:18:01.332 INFO:journalctl@ceph.mon.a.smithi047.stdout: max_auto_readahead_size: 262144 2024-09-17T08:18:01.332 INFO:journalctl@ceph.mon.a.smithi047.stdout: prepopulate_block_cache: 0 2024-09-17T08:18:01.333 INFO:journalctl@ceph.mon.a.smithi047.stdout: initial_auto_readahead_size: 8192 2024-09-17T08:18:01.333 INFO:journalctl@ceph.mon.a.smithi047.stdout: num_file_reads_for_auto_readahead: 2 2024-09-17T08:18:01.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.write_buffer_size: 33554432 2024-09-17T08:18:01.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_write_buffer_number: 2 2024-09-17T08:18:01.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compression: NoCompression 2024-09-17T08:18:01.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.bottommost_compression: Disabled 2024-09-17T08:18:01.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.prefix_extractor: nullptr 2024-09-17T08:18:01.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-17T08:18:01.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.num_levels: 7 2024-09-17T08:18:01.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-17T08:18:01.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-17T08:18:01.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-17T08:18:01.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-17T08:18:01.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-17T08:18:01.334 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-17T08:18:01.334 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-17T08:18:01.334 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-17T08:18:01.334 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-17T08:18:01.334 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-17T08:18:01.334 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-17T08:18:01.334 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-17T08:18:01.334 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-17T08:18:01.334 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compression_opts.level: 32767 2024-09-17T08:18:01.334 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compression_opts.strategy: 0 2024-09-17T08:18:01.334 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-17T08:18:01.334 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-17T08:18:01.334 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-17T08:18:01.334 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-17T08:18:01.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compression_opts.enabled: false 2024-09-17T08:18:01.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-17T08:18:01.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-17T08:18:01.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-17T08:18:01.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-17T08:18:01.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.target_file_size_base: 67108864 2024-09-17T08:18:01.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-17T08:18:01.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-17T08:18:01.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-17T08:18:01.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-17T08:18:01.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-17T08:18:01.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-17T08:18:01.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-17T08:18:01.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-17T08:18:01.336 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-17T08:18:01.336 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-17T08:18:01.336 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-17T08:18:01.336 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-17T08:18:01.336 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-17T08:18:01.336 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-17T08:18:01.336 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.arena_block_size: 1048576 2024-09-17T08:18:01.336 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-17T08:18:01.336 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-17T08:18:01.336 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.disable_auto_compactions: 0 2024-09-17T08:18:01.336 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-17T08:18:01.336 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-17T08:18:01.336 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-17T08:18:01.336 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-17T08:18:01.337 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-17T08:18:01.337 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-17T08:18:01.337 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-17T08:18:01.337 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-17T08:18:01.337 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-17T08:18:01.337 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-17T08:18:01.337 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-17T08:18:01.337 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.inplace_update_support: 0 2024-09-17T08:18:01.337 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-17T08:18:01.337 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-17T08:18:01.337 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-17T08:18:01.337 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-17T08:18:01.337 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.bloom_locality: 0 2024-09-17T08:18:01.337 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.max_successive_merges: 0 2024-09-17T08:18:01.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-17T08:18:01.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.paranoid_file_checks: 0 2024-09-17T08:18:01.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.force_consistency_checks: 1 2024-09-17T08:18:01.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.report_bg_io_stats: 0 2024-09-17T08:18:01.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.ttl: 2592000 2024-09-17T08:18:01.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-17T08:18:01.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-17T08:18:01.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-17T08:18:01.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.enable_blob_files: false 2024-09-17T08:18:01.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.min_blob_size: 0 2024-09-17T08:18:01.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.blob_file_size: 268435456 2024-09-17T08:18:01.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-17T08:18:01.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-17T08:18:01.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-17T08:18:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-17T08:18:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-17T08:18:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.blob_file_starting_level: 0 2024-09-17T08:18:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-17T08:18:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-17T08:18:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-17T08:18:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d1ac55f1-4087-4893-bde2-72df9b628d5f 2024-09-17T08:18:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726561081276500, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-17T08:18:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-17T08:18:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726561081277808, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 80509, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 250, "table_properties": {"data_size": 78670, "index_size": 223, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 9896, "raw_average_key_size": 47, "raw_value_size": 72978, "raw_average_value_size": 349, "num_data_blocks": 10, "num_entries": 209, "num_filter_entries": 209, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726561081, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d1ac55f1-4087-4893-bde2-72df9b628d5f", "db_session_id": "JEUSJ37VFY5VEETFY314", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-17T08:18:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726561081277957, "job": 1, "event": "recovery_finished"} 2024-09-17T08:18:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-17T08:18:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-17T08:18:01.339 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5591049b8e00 2024-09-17T08:18:01.340 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: DB pointer 0x559104aac000 2024-09-17T08:18:01.340 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-17T08:18:01.340 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-17T08:18:01.340 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** DB Stats ** 2024-09-17T08:18:01.340 INFO:journalctl@ceph.mon.a.smithi047.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T08:18:01.340 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-17T08:18:01.340 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T08:18:01.340 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T08:18:01.340 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-17T08:18:01.340 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T08:18:01.340 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T08:18:01.340 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-09-17T08:18:01.340 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** Compaction Stats [default] ** 2024-09-17T08:18:01.341 INFO:journalctl@ceph.mon.a.smithi047.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T08:18:01.341 INFO:journalctl@ceph.mon.a.smithi047.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-17T08:18:01.341 INFO:journalctl@ceph.mon.a.smithi047.stdout: L0 2/0 80.48 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 97.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T08:18:01.341 INFO:journalctl@ceph.mon.a.smithi047.stdout: Sum 2/0 80.48 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 97.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T08:18:01.341 INFO:journalctl@ceph.mon.a.smithi047.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 97.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T08:18:01.341 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-09-17T08:18:01.341 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** Compaction Stats [default] ** 2024-09-17T08:18:01.341 INFO:journalctl@ceph.mon.a.smithi047.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T08:18:01.341 INFO:journalctl@ceph.mon.a.smithi047.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-17T08:18:01.341 INFO:journalctl@ceph.mon.a.smithi047.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 97.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T08:18:01.341 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-09-17T08:18:01.341 INFO:journalctl@ceph.mon.a.smithi047.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-17T08:18:01.341 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-09-17T08:18:01.342 INFO:journalctl@ceph.mon.a.smithi047.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T08:18:01.342 INFO:journalctl@ceph.mon.a.smithi047.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-17T08:18:01.342 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-17T08:18:01.342 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-17T08:18:01.342 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-17T08:18:01.342 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-17T08:18:01.342 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative compaction: 0.00 GB write, 19.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T08:18:01.342 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval compaction: 0.00 GB write, 19.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T08:18:01.342 INFO:journalctl@ceph.mon.a.smithi047.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-17T08:18:01.342 INFO:journalctl@ceph.mon.a.smithi047.stdout: Block cache BinnedLRUCache@0x55910498d350#8 capacity: 512.00 MB usage: 1.17 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-09-17T08:18:01.342 INFO:journalctl@ceph.mon.a.smithi047.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.41 KB,7.7486e-05%) Misc(1,0.00 KB,0%) 2024-09-17T08:18:01.342 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-09-17T08:18:01.342 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-17T08:18:01.343 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: starting mon.a rank 0 at public addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] at bind addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:18:01.343 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: mon.a@-1(???) e1 preinit fsid 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:18:01.343 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: mon.a@-1(???).mds e1 new map 2024-09-17T08:18:01.343 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: mon.a@-1(???).mds e1 print_map 2024-09-17T08:18:01.343 INFO:journalctl@ceph.mon.a.smithi047.stdout: e1 2024-09-17T08:18:01.343 INFO:journalctl@ceph.mon.a.smithi047.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T08:18:01.343 INFO:journalctl@ceph.mon.a.smithi047.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T08:18:01.343 INFO:journalctl@ceph.mon.a.smithi047.stdout: legacy client fscid: -1 2024-09-17T08:18:01.343 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-09-17T08:18:01.343 INFO:journalctl@ceph.mon.a.smithi047.stdout: No filesystems configured 2024-09-17T08:18:01.343 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-17T08:18:01.343 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:18:01.344 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:18:01.344 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:18:01.344 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-17T08:18:01.344 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 bash[25827]: a0d16c90f0857baa4dd2177ab84ed678902f3e968a053395d6b5394c8041c0a3 2024-09-17T08:18:01.344 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 systemd[1]: Started Ceph mon.a for 3a66518a-74cd-11ef-bceb-c7b262605968. 2024-09-17T08:18:01.344 INFO:teuthology.orchestra.run.smithi047.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-17T08:18:01.664 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-17T08:18:01.664 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: monmap e1: 1 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:18:01.664 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: fsmap 2024-09-17T08:18:01.664 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: osdmap e1: 0 total, 0 up, 0 in 2024-09-17T08:18:01.664 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:01 smithi047 ceph-mon[25866]: mgrmap e1: no daemons active 2024-09-17T08:18:02.311 INFO:teuthology.orchestra.run.smithi047.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-17T08:18:02.334 INFO:teuthology.orchestra.run.smithi047.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:18:02.334 INFO:teuthology.orchestra.run.smithi047.stdout:Creating mgr... 2024-09-17T08:18:02.334 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-17T08:18:02.335 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-17T08:18:02.579 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mgr.y 2024-09-17T08:18:02.579 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mgr.y.service: Unit ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mgr.y.service not loaded. 2024-09-17T08:18:02.736 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3a66518a-74cd-11ef-bceb-c7b262605968.target.wants/ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-3a66518a-74cd-11ef-bceb-c7b262605968@.service. 2024-09-17T08:18:02.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:02 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/3362630164' entity='client.admin' 2024-09-17T08:18:03.230 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:03 smithi047 podman[26105]: 2024-09-17 08:18:03.176771581 +0000 UTC m=+0.324270282 container start e29578f4bd1a84766aa2430f114d2b19e323ea3ab1e39c2ca33f63e62a080ed9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-17T08:18:03.230 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:03 smithi047 bash[26105]: e29578f4bd1a84766aa2430f114d2b19e323ea3ab1e39c2ca33f63e62a080ed9 2024-09-17T08:18:03.244 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T08:18:03.244 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-09-17T08:18:03.252 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T08:18:03.252 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-09-17T08:18:03.252 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-09-17T08:18:03.252 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to enable service . firewalld.service is not available 2024-09-17T08:18:03.261 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T08:18:03.261 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-09-17T08:18:03.268 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T08:18:03.269 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-09-17T08:18:03.269 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-09-17T08:18:03.269 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-17T08:18:03.269 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr to start... 2024-09-17T08:18:03.269 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr... 2024-09-17T08:18:03.490 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:03 smithi047 systemd[1]: Started Ceph mgr.y for 3a66518a-74cd-11ef-bceb-c7b262605968. 2024-09-17T08:18:03.490 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:03 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:03.302+0000 7f8e9ea6d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:18:03.490 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:03 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:03.361+0000 7f8e9ea6d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:18:03.750 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:03 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:03.525+0000 7f8e9ea6d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:18:03.750 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:03 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:03.594+0000 7f8e9ea6d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:18:03.750 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:03 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:03.749+0000 7f8e9ea6d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:18:04.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:03 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/280584934' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-17T08:18:04.040 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:03 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:03.899+0000 7f8e9ea6d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:18:04.344 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:04 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:04.226+0000 7f8e9ea6d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:18:04.344 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:04 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:04.289+0000 7f8e9ea6d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:18:04.349 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-17T08:18:04.349 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-09-17T08:18:04.349 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "3a66518a-74cd-11ef-bceb-c7b262605968", 2024-09-17T08:18:04.349 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-09-17T08:18:04.349 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-17T08:18:04.349 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-17T08:18:04.349 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-17T08:18:04.349 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:18:04.349 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-17T08:18:04.349 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-17T08:18:04.349 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-09-17T08:18:04.349 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:18:04.350 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-17T08:18:04.350 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-09-17T08:18:04.350 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:18:04.350 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-17T08:18:04.350 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-17T08:18:04.350 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:18:04.350 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-17T08:18:04.350 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-17T08:18:04.350 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:18:04.350 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-17T08:18:04.350 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:18:04.350 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-17T08:18:04.350 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-17T08:18:04.350 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-17T08:18:04.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-17T08:18:04.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-17T08:18:04.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-17T08:18:04.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:18:04.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-17T08:18:04.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-17T08:18:04.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-17T08:18:04.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-17T08:18:04.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-17T08:18:04.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-17T08:18:04.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-17T08:18:04.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-17T08:18:04.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-17T08:18:04.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:18:04.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-17T08:18:04.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:18:04.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-17T08:18:04.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-17T08:18:04.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:18:04.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-17T08:18:04.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-17T08:18:04.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-17T08:18:04.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-17T08:18:04.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-17T08:18:04.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-17T08:18:04.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-09-17T08:18:04.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:18:04.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T08:18:04.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:18:04.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-17T08:18:04.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:18:04.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-09-17T08:17:56.596134+0000", 2024-09-17T08:18:04.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T08:18:04.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:18:04.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-17T08:18:04.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-09-17T08:18:04.356 INFO:teuthology.orchestra.run.smithi047.stdout:mgr not available, waiting (1/15)... 2024-09-17T08:18:04.789 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:04 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:04.480+0000 7f8e9ea6d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:18:05.789 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:05 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:05.384+0000 7f8e9ea6d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:18:05.789 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:05 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:05.452+0000 7f8e9ea6d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:18:06.462 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:06 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:06.201+0000 7f8e9ea6d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:18:06.462 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:06 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:06.321+0000 7f8e9ea6d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:18:06.771 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:06 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:06.471+0000 7f8e9ea6d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:18:06.771 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:06 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:06.534+0000 7f8e9ea6d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:18:06.771 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:06 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:06.696+0000 7f8e9ea6d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:18:06.771 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:06 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:06.769+0000 7f8e9ea6d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:18:07.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:06 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/2215451990' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-17T08:18:07.041 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:06 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:06.833+0000 7f8e9ea6d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:18:07.041 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:07 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:06.999+0000 7f8e9ea6d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:18:07.347 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:07 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:07.066+0000 7f8e9ea6d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:18:07.347 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:07 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:07.135+0000 7f8e9ea6d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:18:07.347 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:07 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:07.221+0000 7f8e9ea6d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:18:07.347 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:07 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:07.283+0000 7f8e9ea6d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:18:07.347 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:07 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:07.345+0000 7f8e9ea6d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:18:07.407 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-17T08:18:07.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-09-17T08:18:07.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "3a66518a-74cd-11ef-bceb-c7b262605968", 2024-09-17T08:18:07.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-09-17T08:18:07.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-17T08:18:07.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-17T08:18:07.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-17T08:18:07.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:18:07.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-17T08:18:07.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-17T08:18:07.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-09-17T08:18:07.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:18:07.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-17T08:18:07.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-09-17T08:18:07.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:18:07.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-17T08:18:07.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-17T08:18:07.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:18:07.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-17T08:18:07.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-17T08:18:07.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:18:07.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-17T08:18:07.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:18:07.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-17T08:18:07.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-17T08:18:07.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-17T08:18:07.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-17T08:18:07.409 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-17T08:18:07.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-17T08:18:07.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:18:07.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-17T08:18:07.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-17T08:18:07.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-17T08:18:07.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-17T08:18:07.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-17T08:18:07.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-17T08:18:07.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-17T08:18:07.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-17T08:18:07.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-17T08:18:07.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:18:07.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-17T08:18:07.410 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:18:07.411 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-17T08:18:07.411 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-17T08:18:07.411 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:18:07.411 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-17T08:18:07.411 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-17T08:18:07.411 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-17T08:18:07.411 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-17T08:18:07.411 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-17T08:18:07.411 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-17T08:18:07.411 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-09-17T08:18:07.411 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:18:07.411 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T08:18:07.411 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:18:07.411 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-17T08:18:07.411 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:18:07.412 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-09-17T08:17:56.596134+0000", 2024-09-17T08:18:07.412 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T08:18:07.412 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:18:07.412 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-17T08:18:07.412 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-09-17T08:18:07.412 INFO:teuthology.orchestra.run.smithi047.stdout:mgr not available, waiting (2/15)... 2024-09-17T08:18:08.539 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:08 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:08.156+0000 7f8e9ea6d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:18:08.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:08 smithi047 ceph-mon[25866]: Activating manager daemon y 2024-09-17T08:18:08.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:08 smithi047 ceph-mon[25866]: mgrmap e2: y(active, starting, since 0.00333456s) 2024-09-17T08:18:08.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:08 smithi047 ceph-mon[25866]: from='mgr.14100 172.21.15.47:0/18937536' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:18:08.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:08 smithi047 ceph-mon[25866]: from='mgr.14100 172.21.15.47:0/18937536' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:18:08.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:08 smithi047 ceph-mon[25866]: from='mgr.14100 172.21.15.47:0/18937536' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:18:08.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:08 smithi047 ceph-mon[25866]: from='mgr.14100 172.21.15.47:0/18937536' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:18:08.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:08 smithi047 ceph-mon[25866]: from='mgr.14100 172.21.15.47:0/18937536' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:18:08.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:08 smithi047 ceph-mon[25866]: Manager daemon y is now available 2024-09-17T08:18:08.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:08 smithi047 ceph-mon[25866]: from='mgr.14100 172.21.15.47:0/18937536' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:18:08.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:08 smithi047 ceph-mon[25866]: from='mgr.14100 172.21.15.47:0/18937536' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:18:08.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:08 smithi047 ceph-mon[25866]: from='mgr.14100 172.21.15.47:0/18937536' entity='mgr.y' 2024-09-17T08:18:08.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:08 smithi047 ceph-mon[25866]: from='mgr.14100 172.21.15.47:0/18937536' entity='mgr.y' 2024-09-17T08:18:08.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:08 smithi047 ceph-mon[25866]: from='mgr.14100 172.21.15.47:0/18937536' entity='mgr.y' 2024-09-17T08:18:10.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:10 smithi047 ceph-mon[25866]: mgrmap e3: y(active, since 1.00739s) 2024-09-17T08:18:10.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:10 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/3765361991' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-17T08:18:10.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-17T08:18:10.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-09-17T08:18:10.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "3a66518a-74cd-11ef-bceb-c7b262605968", 2024-09-17T08:18:10.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-09-17T08:18:10.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-17T08:18:10.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-17T08:18:10.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-17T08:18:10.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:18:10.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-17T08:18:10.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-17T08:18:10.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-09-17T08:18:10.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:18:10.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-17T08:18:10.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-09-17T08:18:10.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:18:10.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-17T08:18:10.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-17T08:18:10.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:18:10.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-17T08:18:10.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-17T08:18:10.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:18:10.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-17T08:18:10.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:18:10.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-17T08:18:10.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-17T08:18:10.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-17T08:18:10.573 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-17T08:18:10.573 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-17T08:18:10.574 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-17T08:18:10.574 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:18:10.574 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-17T08:18:10.574 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-17T08:18:10.574 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-17T08:18:10.574 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-17T08:18:10.574 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-17T08:18:10.574 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-17T08:18:10.574 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-17T08:18:10.574 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-17T08:18:10.574 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-17T08:18:10.574 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:18:10.574 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-17T08:18:10.574 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:18:10.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-17T08:18:10.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-17T08:18:10.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:18:10.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-17T08:18:10.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-17T08:18:10.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-17T08:18:10.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-17T08:18:10.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-17T08:18:10.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-17T08:18:10.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-09-17T08:18:10.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:18:10.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T08:18:10.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:18:10.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-17T08:18:10.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:18:10.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-09-17T08:17:56.596134+0000", 2024-09-17T08:18:10.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T08:18:10.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:18:10.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-17T08:18:10.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-09-17T08:18:10.576 INFO:teuthology.orchestra.run.smithi047.stdout:mgr is available 2024-09-17T08:18:11.536 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:11 smithi047 ceph-mon[25866]: mgrmap e4: y(active, since 2s) 2024-09-17T08:18:11.782 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-17T08:18:11.782 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [global] 2024-09-17T08:18:11.782 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout fsid = 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:18:11.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-17T08:18:11.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-17T08:18:11.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-17T08:18:11.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-17T08:18:11.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-17T08:18:11.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-17T08:18:11.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-17T08:18:11.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-17T08:18:11.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [osd] 2024-09-17T08:18:11.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-17T08:18:11.783 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-17T08:18:11.783 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling cephadm module... 2024-09-17T08:18:12.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:12 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/2926338159' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-17T08:18:12.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:12 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/2926338159' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-17T08:18:14.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:13 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/3807024233' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-17T08:18:14.040 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:13 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: ignoring --setuser ceph since I am not root 2024-09-17T08:18:14.040 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:13 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: ignoring --setgroup ceph since I am not root 2024-09-17T08:18:14.040 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:13 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:13.878+0000 7f62f88a5040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:18:14.040 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:13 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:13.937+0000 7f62f88a5040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:18:14.294 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:14 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:14.091+0000 7f62f88a5040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:18:14.295 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:14 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:14.160+0000 7f62f88a5040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:18:14.743 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:14 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:14.312+0000 7f62f88a5040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:18:14.743 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:14 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:14.459+0000 7f62f88a5040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:18:15.034 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:14 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/3807024233' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-17T08:18:15.035 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:14 smithi047 ceph-mon[25866]: mgrmap e5: y(active, since 5s) 2024-09-17T08:18:15.035 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:14 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:14.781+0000 7f62f88a5040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:18:15.035 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:14 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:14.843+0000 7f62f88a5040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:18:15.035 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:15 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:15.034+0000 7f62f88a5040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:18:16.277 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:15 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:15.970+0000 7f62f88a5040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:18:16.277 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:16 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:16.040+0000 7f62f88a5040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:18:17.061 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:16 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:16.791+0000 7f62f88a5040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:18:17.061 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:16 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:16.913+0000 7f62f88a5040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:18:17.347 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:17 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:17.060+0000 7f62f88a5040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:18:17.347 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:17 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:17.123+0000 7f62f88a5040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:18:17.348 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:17 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:17.276+0000 7f62f88a5040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:18:17.348 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:17 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:17.346+0000 7f62f88a5040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:18:17.640 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:17 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:17.408+0000 7f62f88a5040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:18:17.640 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:17 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:17.571+0000 7f62f88a5040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:18:17.640 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:17 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:17.639+0000 7f62f88a5040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:18:17.917 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:17 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:17.707+0000 7f62f88a5040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:18:17.917 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:17 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:17.793+0000 7f62f88a5040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:18:17.917 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:17 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:17.855+0000 7f62f88a5040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:18:17.917 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:17 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:17.916+0000 7f62f88a5040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:18:19.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:18 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:18.730+0000 7f62f88a5040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:18:19.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:18 smithi047 ceph-mon[25866]: Active manager daemon y restarted 2024-09-17T08:18:19.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:18 smithi047 ceph-mon[25866]: Activating manager daemon y 2024-09-17T08:18:19.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:18 smithi047 ceph-mon[25866]: osdmap e2: 0 total, 0 up, 0 in 2024-09-17T08:18:19.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:18 smithi047 ceph-mon[25866]: mgrmap e6: y(active, starting, since 0.00312463s) 2024-09-17T08:18:19.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:18 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:18:19.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:18 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:18:19.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:18 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:18:19.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:18 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:18:19.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:18 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:18:19.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:18 smithi047 ceph-mon[25866]: Manager daemon y is now available 2024-09-17T08:18:19.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:18 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' 2024-09-17T08:18:19.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:18 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' 2024-09-17T08:18:19.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:18 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:18:19.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:18 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:18:19.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:18 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:18:19.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:18 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:18:20.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:19 smithi047 ceph-mon[25866]: Found migration_current of "None". Setting to last migration. 2024-09-17T08:18:20.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:19 smithi047 ceph-mon[25866]: mgrmap e7: y(active, since 1.0054s) 2024-09-17T08:18:21.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' 2024-09-17T08:18:21.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' 2024-09-17T08:18:22.272 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:21.847+0000 7f6260585640 -1 log_channel(cephadm) log [ERR] : [17/Sep/2024:08:18:21] ENGINE Error in HTTPServer.serve 2024-09-17T08:18:22.273 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: Traceback (most recent call last): 2024-09-17T08:18:22.273 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:18:22.273 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: self._connections.run(self.expiration_interval) 2024-09-17T08:18:22.273 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:18:22.273 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: self._run(expiration_interval) 2024-09-17T08:18:22.273 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:18:22.273 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:18:22.273 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:18:22.273 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:18:22.273 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:18:22.273 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: s = self.context.wrap_socket( 2024-09-17T08:18:22.273 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:18:22.274 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: return self.sslsocket_class._create( 2024-09-17T08:18:22.274 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:18:22.274 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: self.do_handshake() 2024-09-17T08:18:22.274 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:18:22.274 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: self._sslobj.do_handshake() 2024-09-17T08:18:22.274 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:18:22.274 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:21 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:22.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:22 smithi047 ceph-mon[25866]: mgrmap e8: y(active, since 2s) 2024-09-17T08:18:22.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:22 smithi047 ceph-mon[25866]: [17/Sep/2024:08:18:21] ENGINE Bus STARTING 2024-09-17T08:18:22.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:22 smithi047 ceph-mon[25866]: [17/Sep/2024:08:18:21] ENGINE Serving on http://172.21.15.47:8765 2024-09-17T08:18:22.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:22 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:18:23.509 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:23 smithi047 ceph-mon[25866]: [17/Sep/2024:08:18:21] ENGINE Serving on https://172.21.15.47:7150 2024-09-17T08:18:23.509 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:23 smithi047 ceph-mon[25866]: [17/Sep/2024:08:18:21] ENGINE Bus STARTED 2024-09-17T08:18:23.509 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:23 smithi047 ceph-mon[25866]: [17/Sep/2024:08:18:21] ENGINE Error in HTTPServer.serve 2024-09-17T08:18:23.509 INFO:journalctl@ceph.mon.a.smithi047.stdout: Traceback (most recent call last): 2024-09-17T08:18:23.509 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:18:23.510 INFO:journalctl@ceph.mon.a.smithi047.stdout: self._connections.run(self.expiration_interval) 2024-09-17T08:18:23.510 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:18:23.510 INFO:journalctl@ceph.mon.a.smithi047.stdout: self._run(expiration_interval) 2024-09-17T08:18:23.510 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:18:23.510 INFO:journalctl@ceph.mon.a.smithi047.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:18:23.510 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:18:23.510 INFO:journalctl@ceph.mon.a.smithi047.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:18:23.510 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:18:23.510 INFO:journalctl@ceph.mon.a.smithi047.stdout: s = self.context.wrap_socket( 2024-09-17T08:18:23.510 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:18:23.510 INFO:journalctl@ceph.mon.a.smithi047.stdout: return self.sslsocket_class._create( 2024-09-17T08:18:23.510 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:18:23.510 INFO:journalctl@ceph.mon.a.smithi047.stdout: self.do_handshake() 2024-09-17T08:18:23.511 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:18:23.511 INFO:journalctl@ceph.mon.a.smithi047.stdout: self._sslobj.do_handshake() 2024-09-17T08:18:23.511 INFO:journalctl@ceph.mon.a.smithi047.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:18:23.511 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:23 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/1151455539' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-17T08:18:23.737 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-09-17T08:18:23.737 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 8, 2024-09-17T08:18:23.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-17T08:18:23.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-17T08:18:23.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-17T08:18:23.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-09-17T08:18:23.738 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for the mgr to restart... 2024-09-17T08:18:23.738 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr epoch 8... 2024-09-17T08:18:24.718 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-09-17T08:18:24.718 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 8, 2024-09-17T08:18:24.718 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-17T08:18:24.718 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-09-17T08:18:24.718 INFO:teuthology.orchestra.run.smithi047.stdout:mgr epoch 8 is available 2024-09-17T08:18:24.719 INFO:teuthology.orchestra.run.smithi047.stdout:Setting orchestrator backend to cephadm... 2024-09-17T08:18:25.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:24 smithi047 ceph-mon[25866]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-17T08:18:25.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:24 smithi047 ceph-mon[25866]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-17T08:18:27.281 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:27 smithi047 ceph-mon[25866]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:18:27.281 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:27 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' 2024-09-17T08:18:27.281 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:27 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:18:27.815 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-17T08:18:27.815 INFO:teuthology.orchestra.run.smithi047.stdout:Generating ssh key... 2024-09-17T08:18:28.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:28 smithi047 ceph-mon[25866]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:18:29.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:29 smithi047 ceph-mon[25866]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:18:29.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:29 smithi047 ceph-mon[25866]: Generating ssh key... 2024-09-17T08:18:29.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:29 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' 2024-09-17T08:18:29.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:29 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' 2024-09-17T08:18:29.185 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:28 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: Generating public/private rsa key pair. 2024-09-17T08:18:29.185 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:28 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: Your identification has been saved in /tmp/tmpd4ubh5qf/key 2024-09-17T08:18:29.185 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:28 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: Your public key has been saved in /tmp/tmpd4ubh5qf/key.pub 2024-09-17T08:18:29.185 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:28 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: The key fingerprint is: 2024-09-17T08:18:29.186 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:28 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: SHA256:3SBxgeRZz51M97i50VSZFwuyY1Yuot4UvTpEmHB1gkU ceph-3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:18:29.186 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:28 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: The key's randomart image is: 2024-09-17T08:18:29.186 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:28 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: +---[RSA 3072]----+ 2024-09-17T08:18:29.186 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:28 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: | . oBE++ o..*| 2024-09-17T08:18:29.186 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:28 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: | oo+=+o=+.*=| 2024-09-17T08:18:29.186 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:28 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: | o++.Bo.=.+| 2024-09-17T08:18:29.186 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:28 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: | oo=o+ = | 2024-09-17T08:18:29.186 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:28 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: | .So... + .| 2024-09-17T08:18:29.186 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:28 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: | . + . o | 2024-09-17T08:18:29.187 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:28 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: | . + . | 2024-09-17T08:18:29.187 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:28 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: | . | 2024-09-17T08:18:29.187 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:28 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: | | 2024-09-17T08:18:29.187 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:28 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: +----[SHA256]-----+ 2024-09-17T08:18:30.526 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkxsT6VlNWmi+qioF/gNHpdBiS+Wjysbu55QlXHHDvDgtibb3XbsGiRpY1stPzWx99h1QWmKCUnL2BefxffGTbm7Go5x0auNoFU/dKBIAG1VGk631uOzJT9PY56WYISsQHxg7p5lvPQQiCTdrAl05E8hR4fZro9Aoy9a8V4V5zxfx1rzDp3+VjC6kaVWQYm3KZXBicL5DMBDtxCN7YCS8LkzFRwusT7trdevMMxuKkvpC9kaiEo0lrYm0YFOXkLJ2AZFypl2P6yzLilgXE01PynDue0J6xIYCGdaZXhQPO2g2oI0/pB2MX3NAUtyxZffivu5li79iYgcXu+mc8NN+4xm+PjG+ZJp7mkGjW3FSZB+cU175V6c6Fz69VSnQsFJ2b3PLhXbLWJBat8OwIuE2jwNLJjYy0W/iksYK8nqS+WgjN/jyZ33jgxQsC2mufA9iqH0huON19G2AcPtKrjNAeV5OwsRy5ECKK65J8ROgrycJAVK3eV3HvbmIu4SpKrDs= ceph-3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:18:30.526 INFO:teuthology.orchestra.run.smithi047.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-17T08:18:30.526 INFO:teuthology.orchestra.run.smithi047.stdout:Adding key to root@localhost authorized_keys... 2024-09-17T08:18:30.536 INFO:teuthology.orchestra.run.smithi047.stdout:Adding host smithi047... 2024-09-17T08:18:30.951 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:30 smithi047 ceph-mon[25866]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:18:32.164 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:31 smithi047 ceph-mon[25866]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi047", "addr": "172.21.15.47", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:18:33.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:32 smithi047 ceph-mon[25866]: Deploying cephadm binary to smithi047 2024-09-17T08:18:35.059 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Added host 'smithi047' with addr '172.21.15.47' 2024-09-17T08:18:35.059 INFO:teuthology.orchestra.run.smithi047.stdout:Deploying unmanaged mon service... 2024-09-17T08:18:35.719 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:35 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' 2024-09-17T08:18:35.719 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:35 smithi047 ceph-mon[25866]: Added host smithi047 2024-09-17T08:18:35.719 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:35 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:18:36.526 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-17T08:18:36.526 INFO:teuthology.orchestra.run.smithi047.stdout:Deploying unmanaged mgr service... 2024-09-17T08:18:37.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:36 smithi047 ceph-mon[25866]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:18:37.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:36 smithi047 ceph-mon[25866]: Saving service mon spec with placement count:5 2024-09-17T08:18:37.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:36 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' 2024-09-17T08:18:38.051 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-17T08:18:38.414 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:38 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' 2024-09-17T08:18:38.414 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:38 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' 2024-09-17T08:18:38.414 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:38 smithi047 ceph-mon[25866]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:18:38.414 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:38 smithi047 ceph-mon[25866]: Saving service mgr spec with placement count:2 2024-09-17T08:18:38.414 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:38 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' 2024-09-17T08:18:39.933 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:39 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/2176201450' entity='client.admin' 2024-09-17T08:18:39.933 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:39 smithi047 ceph-mon[25866]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:18:39.933 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:39 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' 2024-09-17T08:18:41.217 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling the dashboard module... 2024-09-17T08:18:41.843 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:41 smithi047 ceph-mon[25866]: from='mgr.14118 172.21.15.47:0/1608845808' entity='mgr.y' 2024-09-17T08:18:41.843 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:41 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/287567835' entity='client.admin' 2024-09-17T08:18:41.843 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:41 smithi047 ceph-mon[25866]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:18:42.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:42 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/2785498209' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-17T08:18:42.977 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:42 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: ignoring --setuser ceph since I am not root 2024-09-17T08:18:42.977 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:42 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: ignoring --setgroup ceph since I am not root 2024-09-17T08:18:42.977 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:42 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:42.761+0000 7f1c0ca29040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:18:42.978 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:42 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:42.819+0000 7f1c0ca29040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:18:42.978 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:42 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:42.976+0000 7f1c0ca29040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:18:43.285 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:43 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:43.045+0000 7f1c0ca29040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:18:43.285 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:43 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:43.196+0000 7f1c0ca29040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:18:43.539 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:43 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:43.346+0000 7f1c0ca29040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:18:43.930 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:43 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/2785498209' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-17T08:18:43.930 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:43 smithi047 ceph-mon[25866]: mgrmap e9: y(active, since 23s) 2024-09-17T08:18:43.930 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:43 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:43.671+0000 7f1c0ca29040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:18:43.930 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:43 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:43.739+0000 7f1c0ca29040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:18:43.930 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:43 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:43.929+0000 7f1c0ca29040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:18:44.552 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-09-17T08:18:44.552 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-17T08:18:44.552 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-17T08:18:44.552 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-17T08:18:44.552 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-17T08:18:44.552 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-09-17T08:18:44.552 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for the mgr to restart... 2024-09-17T08:18:44.552 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr epoch 9... 2024-09-17T08:18:44.801 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:44 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/3944589556' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-17T08:18:45.289 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:44 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:44.852+0000 7f1c0ca29040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:18:45.290 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:44 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:44.920+0000 7f1c0ca29040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:18:45.932 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:45 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:45.664+0000 7f1c0ca29040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:18:45.933 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:45 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:45.783+0000 7f1c0ca29040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:18:46.226 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:45 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:45.932+0000 7f1c0ca29040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:18:46.226 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:45 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:45.996+0000 7f1c0ca29040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:18:46.226 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:46 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:46.151+0000 7f1c0ca29040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:18:46.226 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:46 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:46.224+0000 7f1c0ca29040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:18:46.523 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:46 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:46.289+0000 7f1c0ca29040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:18:46.524 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:46 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:46.454+0000 7f1c0ca29040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:18:46.524 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:46 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:46.523+0000 7f1c0ca29040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:18:46.789 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:46 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:46.592+0000 7f1c0ca29040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:18:46.790 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:46 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:46.678+0000 7f1c0ca29040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:18:46.790 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:46 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:46.740+0000 7f1c0ca29040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:18:47.289 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:46 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:46.801+0000 7f1c0ca29040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:18:47.881 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:47 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:47.612+0000 7f1c0ca29040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:18:47.882 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:47 smithi047 ceph-mon[25866]: Active manager daemon y restarted 2024-09-17T08:18:47.882 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:47 smithi047 ceph-mon[25866]: Activating manager daemon y 2024-09-17T08:18:47.882 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:47 smithi047 ceph-mon[25866]: osdmap e3: 0 total, 0 up, 0 in 2024-09-17T08:18:47.882 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:47 smithi047 ceph-mon[25866]: mgrmap e10: y(active, starting, since 0.0031551s) 2024-09-17T08:18:47.882 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:47 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:18:47.882 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:47 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:18:47.882 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:47 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:18:47.882 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:47 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:18:47.882 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:47 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:18:47.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:47 smithi047 ceph-mon[25866]: Manager daemon y is now available 2024-09-17T08:18:47.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:47 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:18:47.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:47 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:18:47.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:47 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:18:49.462 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-09-17T08:18:49.463 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-17T08:18:49.463 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-17T08:18:49.464 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-09-17T08:18:49.464 INFO:teuthology.orchestra.run.smithi047.stdout:mgr epoch 9 is available 2024-09-17T08:18:49.464 INFO:teuthology.orchestra.run.smithi047.stdout:Generating a dashboard self-signed certificate... 2024-09-17T08:18:49.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-mon[25866]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-17T08:18:49.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-mon[25866]: mgrmap e11: y(active, since 1.00633s) 2024-09-17T08:18:49.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-mon[25866]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-17T08:18:49.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:18:49.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:18:49.700 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:49.374+0000 7f1b72e73640 -1 log_channel(cephadm) log [ERR] : [17/Sep/2024:08:18:49] ENGINE Error in HTTPServer.serve 2024-09-17T08:18:49.700 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: Traceback (most recent call last): 2024-09-17T08:18:49.701 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:18:49.701 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: self._connections.run(self.expiration_interval) 2024-09-17T08:18:49.701 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:18:49.701 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: self._run(expiration_interval) 2024-09-17T08:18:49.701 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:18:49.701 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:18:49.701 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:18:49.701 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:18:49.701 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:18:49.701 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: s = self.context.wrap_socket( 2024-09-17T08:18:49.701 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:18:49.702 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: return self.sslsocket_class._create( 2024-09-17T08:18:49.702 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:18:49.702 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: self.do_handshake() 2024-09-17T08:18:49.702 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:18:49.702 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: self._sslobj.do_handshake() 2024-09-17T08:18:49.702 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:18:49.702 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:18:49 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:18:50.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:50 smithi047 ceph-mon[25866]: [17/Sep/2024:08:18:49] ENGINE Bus STARTING 2024-09-17T08:18:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:50 smithi047 ceph-mon[25866]: [17/Sep/2024:08:18:49] ENGINE Serving on http://172.21.15.47:8765 2024-09-17T08:18:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:50 smithi047 ceph-mon[25866]: [17/Sep/2024:08:18:49] ENGINE Serving on https://172.21.15.47:7150 2024-09-17T08:18:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:50 smithi047 ceph-mon[25866]: [17/Sep/2024:08:18:49] ENGINE Bus STARTED 2024-09-17T08:18:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:50 smithi047 ceph-mon[25866]: [17/Sep/2024:08:18:49] ENGINE Error in HTTPServer.serve 2024-09-17T08:18:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout: Traceback (most recent call last): 2024-09-17T08:18:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:18:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout: self._connections.run(self.expiration_interval) 2024-09-17T08:18:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:18:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout: self._run(expiration_interval) 2024-09-17T08:18:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:18:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:18:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:18:50.791 INFO:journalctl@ceph.mon.a.smithi047.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:18:50.791 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:18:50.791 INFO:journalctl@ceph.mon.a.smithi047.stdout: s = self.context.wrap_socket( 2024-09-17T08:18:50.791 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:18:50.791 INFO:journalctl@ceph.mon.a.smithi047.stdout: return self.sslsocket_class._create( 2024-09-17T08:18:50.791 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:18:50.791 INFO:journalctl@ceph.mon.a.smithi047.stdout: self.do_handshake() 2024-09-17T08:18:50.791 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:18:50.791 INFO:journalctl@ceph.mon.a.smithi047.stdout: self._sslobj.do_handshake() 2024-09-17T08:18:50.791 INFO:journalctl@ceph.mon.a.smithi047.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:18:50.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:50 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:18:50.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:50 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:18:50.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:50 smithi047 ceph-mon[25866]: mgrmap e12: y(active, since 2s) 2024-09-17T08:18:51.448 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-17T08:18:51.448 INFO:teuthology.orchestra.run.smithi047.stdout:Creating initial admin user... 2024-09-17T08:18:51.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:51 smithi047 ceph-mon[25866]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:18:52.962 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$84fymSUZi3ACnGLEfIUx2OkmeKP3h1E4tJ1.Xo7t0l2Zq5X4N1Y7y", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726561132, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-17T08:18:52.962 INFO:teuthology.orchestra.run.smithi047.stdout:Fetching dashboard port number... 2024-09-17T08:18:53.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:53 smithi047 ceph-mon[25866]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:18:53.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:53 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:18:54.076 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 8443 2024-09-17T08:18:54.086 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T08:18:54.086 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-09-17T08:18:54.094 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T08:18:54.094 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-09-17T08:18:54.094 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-09-17T08:18:54.094 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-17T08:18:54.096 INFO:teuthology.orchestra.run.smithi047.stdout:Ceph Dashboard is now available at: 2024-09-17T08:18:54.097 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:18:54.097 INFO:teuthology.orchestra.run.smithi047.stdout: URL: https://smithi047.front.sepia.ceph.com:8443/ 2024-09-17T08:18:54.097 INFO:teuthology.orchestra.run.smithi047.stdout: User: admin 2024-09-17T08:18:54.097 INFO:teuthology.orchestra.run.smithi047.stdout: Password: lzeuqjvmaw 2024-09-17T08:18:54.097 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:18:54.097 INFO:teuthology.orchestra.run.smithi047.stdout:Saving cluster configuration to /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config directory 2024-09-17T08:18:54.097 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling autotune for osd_memory_target 2024-09-17T08:18:54.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:54 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/1148124705' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-17T08:18:55.665 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:55 smithi047 ceph-mon[25866]: mgrmap e13: y(active, since 6s) 2024-09-17T08:18:56.522 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-17T08:18:56.522 INFO:teuthology.orchestra.run.smithi047.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-17T08:18:56.522 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:18:56.522 INFO:teuthology.orchestra.run.smithi047.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:18:56.522 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:18:56.522 INFO:teuthology.orchestra.run.smithi047.stdout:Or, if you are only running a single cluster on this host: 2024-09-17T08:18:56.522 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:18:56.522 INFO:teuthology.orchestra.run.smithi047.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-17T08:18:56.522 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:18:56.522 INFO:teuthology.orchestra.run.smithi047.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-17T08:18:56.522 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:18:56.522 INFO:teuthology.orchestra.run.smithi047.stdout: ceph telemetry on 2024-09-17T08:18:56.523 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:18:56.523 INFO:teuthology.orchestra.run.smithi047.stdout:For more information see: 2024-09-17T08:18:56.523 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:18:56.523 INFO:teuthology.orchestra.run.smithi047.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-17T08:18:56.523 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:18:56.523 INFO:teuthology.orchestra.run.smithi047.stdout:Bootstrap complete. 2024-09-17T08:18:56.560 INFO:tasks.cephadm:Fetching config... 2024-09-17T08:18:56.560 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-17T08:18:56.560 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-17T08:18:56.577 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-17T08:18:56.577 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-17T08:18:56.577 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-17T08:18:56.634 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-17T08:18:56.634 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-17T08:18:56.634 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-17T08:18:56.699 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-17T08:18:56.699 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-17T08:18:56.699 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-17T08:18:56.756 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-17T08:18:56.757 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkxsT6VlNWmi+qioF/gNHpdBiS+Wjysbu55QlXHHDvDgtibb3XbsGiRpY1stPzWx99h1QWmKCUnL2BefxffGTbm7Go5x0auNoFU/dKBIAG1VGk631uOzJT9PY56WYISsQHxg7p5lvPQQiCTdrAl05E8hR4fZro9Aoy9a8V4V5zxfx1rzDp3+VjC6kaVWQYm3KZXBicL5DMBDtxCN7YCS8LkzFRwusT7trdevMMxuKkvpC9kaiEo0lrYm0YFOXkLJ2AZFypl2P6yzLilgXE01PynDue0J6xIYCGdaZXhQPO2g2oI0/pB2MX3NAUtyxZffivu5li79iYgcXu+mc8NN+4xm+PjG+ZJp7mkGjW3FSZB+cU175V6c6Fz69VSnQsFJ2b3PLhXbLWJBat8OwIuE2jwNLJjYy0W/iksYK8nqS+WgjN/jyZ33jgxQsC2mufA9iqH0huON19G2AcPtKrjNAeV5OwsRy5ECKK65J8ROgrycJAVK3eV3HvbmIu4SpKrDs= ceph-3a66518a-74cd-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-17T08:18:56.831 INFO:teuthology.orchestra.run.smithi047.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkxsT6VlNWmi+qioF/gNHpdBiS+Wjysbu55QlXHHDvDgtibb3XbsGiRpY1stPzWx99h1QWmKCUnL2BefxffGTbm7Go5x0auNoFU/dKBIAG1VGk631uOzJT9PY56WYISsQHxg7p5lvPQQiCTdrAl05E8hR4fZro9Aoy9a8V4V5zxfx1rzDp3+VjC6kaVWQYm3KZXBicL5DMBDtxCN7YCS8LkzFRwusT7trdevMMxuKkvpC9kaiEo0lrYm0YFOXkLJ2AZFypl2P6yzLilgXE01PynDue0J6xIYCGdaZXhQPO2g2oI0/pB2MX3NAUtyxZffivu5li79iYgcXu+mc8NN+4xm+PjG+ZJp7mkGjW3FSZB+cU175V6c6Fz69VSnQsFJ2b3PLhXbLWJBat8OwIuE2jwNLJjYy0W/iksYK8nqS+WgjN/jyZ33jgxQsC2mufA9iqH0huON19G2AcPtKrjNAeV5OwsRy5ECKK65J8ROgrycJAVK3eV3HvbmIu4SpKrDs= ceph-3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:18:56.845 DEBUG:teuthology.orchestra.run.smithi193:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkxsT6VlNWmi+qioF/gNHpdBiS+Wjysbu55QlXHHDvDgtibb3XbsGiRpY1stPzWx99h1QWmKCUnL2BefxffGTbm7Go5x0auNoFU/dKBIAG1VGk631uOzJT9PY56WYISsQHxg7p5lvPQQiCTdrAl05E8hR4fZro9Aoy9a8V4V5zxfx1rzDp3+VjC6kaVWQYm3KZXBicL5DMBDtxCN7YCS8LkzFRwusT7trdevMMxuKkvpC9kaiEo0lrYm0YFOXkLJ2AZFypl2P6yzLilgXE01PynDue0J6xIYCGdaZXhQPO2g2oI0/pB2MX3NAUtyxZffivu5li79iYgcXu+mc8NN+4xm+PjG+ZJp7mkGjW3FSZB+cU175V6c6Fz69VSnQsFJ2b3PLhXbLWJBat8OwIuE2jwNLJjYy0W/iksYK8nqS+WgjN/jyZ33jgxQsC2mufA9iqH0huON19G2AcPtKrjNAeV5OwsRy5ECKK65J8ROgrycJAVK3eV3HvbmIu4SpKrDs= ceph-3a66518a-74cd-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-17T08:18:56.897 INFO:teuthology.orchestra.run.smithi193.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkxsT6VlNWmi+qioF/gNHpdBiS+Wjysbu55QlXHHDvDgtibb3XbsGiRpY1stPzWx99h1QWmKCUnL2BefxffGTbm7Go5x0auNoFU/dKBIAG1VGk631uOzJT9PY56WYISsQHxg7p5lvPQQiCTdrAl05E8hR4fZro9Aoy9a8V4V5zxfx1rzDp3+VjC6kaVWQYm3KZXBicL5DMBDtxCN7YCS8LkzFRwusT7trdevMMxuKkvpC9kaiEo0lrYm0YFOXkLJ2AZFypl2P6yzLilgXE01PynDue0J6xIYCGdaZXhQPO2g2oI0/pB2MX3NAUtyxZffivu5li79iYgcXu+mc8NN+4xm+PjG+ZJp7mkGjW3FSZB+cU175V6c6Fz69VSnQsFJ2b3PLhXbLWJBat8OwIuE2jwNLJjYy0W/iksYK8nqS+WgjN/jyZ33jgxQsC2mufA9iqH0huON19G2AcPtKrjNAeV5OwsRy5ECKK65J8ROgrycJAVK3eV3HvbmIu4SpKrDs= ceph-3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:18:56.909 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkxsT6VlNWmi+qioF/gNHpdBiS+Wjysbu55QlXHHDvDgtibb3XbsGiRpY1stPzWx99h1QWmKCUnL2BefxffGTbm7Go5x0auNoFU/dKBIAG1VGk631uOzJT9PY56WYISsQHxg7p5lvPQQiCTdrAl05E8hR4fZro9Aoy9a8V4V5zxfx1rzDp3+VjC6kaVWQYm3KZXBicL5DMBDtxCN7YCS8LkzFRwusT7trdevMMxuKkvpC9kaiEo0lrYm0YFOXkLJ2AZFypl2P6yzLilgXE01PynDue0J6xIYCGdaZXhQPO2g2oI0/pB2MX3NAUtyxZffivu5li79iYgcXu+mc8NN+4xm+PjG+ZJp7mkGjW3FSZB+cU175V6c6Fz69VSnQsFJ2b3PLhXbLWJBat8OwIuE2jwNLJjYy0W/iksYK8nqS+WgjN/jyZ33jgxQsC2mufA9iqH0huON19G2AcPtKrjNAeV5OwsRy5ECKK65J8ROgrycJAVK3eV3HvbmIu4SpKrDs= ceph-3a66518a-74cd-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-17T08:18:57.051 INFO:teuthology.orchestra.run.smithi203.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkxsT6VlNWmi+qioF/gNHpdBiS+Wjysbu55QlXHHDvDgtibb3XbsGiRpY1stPzWx99h1QWmKCUnL2BefxffGTbm7Go5x0auNoFU/dKBIAG1VGk631uOzJT9PY56WYISsQHxg7p5lvPQQiCTdrAl05E8hR4fZro9Aoy9a8V4V5zxfx1rzDp3+VjC6kaVWQYm3KZXBicL5DMBDtxCN7YCS8LkzFRwusT7trdevMMxuKkvpC9kaiEo0lrYm0YFOXkLJ2AZFypl2P6yzLilgXE01PynDue0J6xIYCGdaZXhQPO2g2oI0/pB2MX3NAUtyxZffivu5li79iYgcXu+mc8NN+4xm+PjG+ZJp7mkGjW3FSZB+cU175V6c6Fz69VSnQsFJ2b3PLhXbLWJBat8OwIuE2jwNLJjYy0W/iksYK8nqS+WgjN/jyZ33jgxQsC2mufA9iqH0huON19G2AcPtKrjNAeV5OwsRy5ECKK65J8ROgrycJAVK3eV3HvbmIu4SpKrDs= ceph-3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:18:57.064 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-17T08:18:57.089 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:18:56 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/957214350' entity='client.admin' 2024-09-17T08:18:57.266 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:00.282 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:00 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:00.283 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:00 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:00.283 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:00 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi047", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:19:00.283 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:00 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:00.283 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:00 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:00.283 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:00 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:00.283 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:00 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/4220800280' entity='client.admin' 2024-09-17T08:19:00.283 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:00 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:00.283 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:00 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:00.622 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-17T08:19:00.622 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-17T08:19:01.269 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:03.615 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi193 2024-09-17T08:19:03.615 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-17T08:19:03.615 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.conf 2024-09-17T08:19:03.635 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-17T08:19:03.635 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:19:03.693 INFO:tasks.cephadm:Adding host smithi193 to orchestrator... 2024-09-17T08:19:03.693 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph orch host add smithi193 2024-09-17T08:19:03.897 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:04.253 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:03 smithi047 ceph-mon[25866]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:19:04.253 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:03 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:04.253 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:03 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:04.254 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:03 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:04.254 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:03 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:19:04.254 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:03 smithi047 ceph-mon[25866]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-17T08:19:04.254 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:03 smithi047 ceph-mon[25866]: Updating smithi047:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:19:04.254 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:03 smithi047 ceph-mon[25866]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:19:04.254 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:03 smithi047 ceph-mon[25866]: Updating smithi047:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:19:04.254 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:03 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:04.254 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:03 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:04.254 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:03 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:06.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:05 smithi047 ceph-mon[25866]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:19:08.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:07 smithi047 ceph-mon[25866]: Deploying cephadm binary to smithi193 2024-09-17T08:19:08.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:07 smithi047 ceph-mon[25866]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:08.881 INFO:teuthology.orchestra.run.smithi047.stdout:Added host 'smithi193' with addr '172.21.15.193' 2024-09-17T08:19:09.475 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-17T08:19:09.670 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:10.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:09 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:10.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:09 smithi047 ceph-mon[25866]: Added host smithi193 2024-09-17T08:19:10.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:09 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:10.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:09 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:10.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:09 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:10.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:09 smithi047 ceph-mon[25866]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:11.193 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:19:11.194 INFO:teuthology.orchestra.run.smithi047.stdout:[{"addr": "172.21.15.47", "hostname": "smithi047", "labels": [], "status": ""}, {"addr": "172.21.15.193", "hostname": "smithi193", "labels": [], "status": ""}] 2024-09-17T08:19:11.490 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:11 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:11.808 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi203 2024-09-17T08:19:11.808 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-17T08:19:11.808 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/etc/ceph/ceph.conf 2024-09-17T08:19:11.831 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-17T08:19:11.831 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:19:11.889 INFO:tasks.cephadm:Adding host smithi203 to orchestrator... 2024-09-17T08:19:11.889 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph orch host add smithi203 2024-09-17T08:19:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:12.443 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:12 smithi047 ceph-mon[25866]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:19:12.443 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:12 smithi047 ceph-mon[25866]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:14.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:14 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:14.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:14 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:14.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:14 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:14.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:14 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:14.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:14 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:19:14.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:14 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:14.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:14 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:19:14.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:14 smithi047 ceph-mon[25866]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-17T08:19:14.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:14 smithi047 ceph-mon[25866]: Updating smithi193:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:19:14.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:14 smithi047 ceph-mon[25866]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:19:14.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:14 smithi047 ceph-mon[25866]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:14.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:14 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:14.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:14 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:14.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:14 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:15.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:15 smithi047 ceph-mon[25866]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:19:15.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:15 smithi047 ceph-mon[25866]: Updating smithi193:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:19:16.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:16 smithi047 ceph-mon[25866]: Deploying cephadm binary to smithi203 2024-09-17T08:19:16.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:16 smithi047 ceph-mon[25866]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:17.057 INFO:teuthology.orchestra.run.smithi047.stdout:Added host 'smithi203' with addr '172.21.15.203' 2024-09-17T08:19:17.660 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-17T08:19:17.858 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:18.231 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:18 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:18.231 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:18 smithi047 ceph-mon[25866]: Added host smithi203 2024-09-17T08:19:18.231 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:18 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:18.231 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:18 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:18.231 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:18 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:18.231 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:18 smithi047 ceph-mon[25866]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:19.449 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:19:19.450 INFO:teuthology.orchestra.run.smithi047.stdout:[{"addr": "172.21.15.47", "hostname": "smithi047", "labels": [], "status": ""}, {"addr": "172.21.15.193", "hostname": "smithi193", "labels": [], "status": ""}, {"addr": "172.21.15.203", "hostname": "smithi203", "labels": [], "status": ""}] 2024-09-17T08:19:19.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:19 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:19.953 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-17T08:19:19.953 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-17T08:19:20.149 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:20.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:20 smithi047 ceph-mon[25866]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:19:20.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:20 smithi047 ceph-mon[25866]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:22.096 INFO:teuthology.orchestra.run.smithi047.stderr:adjusted tunables profile to default 2024-09-17T08:19:22.392 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:22 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:22.393 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:22 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:22.393 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:22 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:22.393 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:22 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:22.393 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:22 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:19:22.393 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:22 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:22.393 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:22 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:19:22.393 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:22 smithi047 ceph-mon[25866]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-17T08:19:22.394 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:22 smithi047 ceph-mon[25866]: Updating smithi203:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:19:22.394 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:22 smithi047 ceph-mon[25866]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:19:22.394 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:22 smithi047 ceph-mon[25866]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:22.394 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:22 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/1030672606' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-17T08:19:22.394 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:22 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:22.394 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:22 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:22.394 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:22 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:22.691 INFO:tasks.cephadm:Adding mon.a on smithi047 2024-09-17T08:19:22.691 INFO:tasks.cephadm:Adding mon.b on smithi193 2024-09-17T08:19:22.691 INFO:tasks.cephadm:Adding mon.c on smithi203 2024-09-17T08:19:22.691 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi047:172.21.15.47=a;smithi193:172.21.15.193=b;smithi203:172.21.15.203=c' 2024-09-17T08:19:22.876 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:19:23.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:23 smithi047 ceph-mon[25866]: Updating smithi203:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:19:23.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:23 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/1030672606' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-17T08:19:23.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:23 smithi047 ceph-mon[25866]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:19:24.429 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mon update... 2024-09-17T08:19:24.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:24 smithi047 ceph-mon[25866]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:25.062 DEBUG:teuthology.orchestra.run.smithi193:mon.b> sudo journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mon.b.service 2024-09-17T08:19:25.065 DEBUG:teuthology.orchestra.run.smithi203:mon.c> sudo journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mon.c.service 2024-09-17T08:19:25.067 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-17T08:19:25.067 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-17T08:19:25.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:25 smithi047 ceph-mon[25866]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi193:172.21.15.193=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:19:25.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:25 smithi047 ceph-mon[25866]: Saving service mon spec with placement smithi047:172.21.15.47=a;smithi193:172.21.15.193=b;smithi203:172.21.15.203=c;count:3 2024-09-17T08:19:25.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:25 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:25.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:25 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:25.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:25 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:25.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:25 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:19:25.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:25 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:25.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:25 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:19:25.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:25 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:25.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:25 smithi047 ceph-mon[25866]: Deploying daemon mon.c on smithi203 2024-09-17T08:19:26.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:26 smithi047 ceph-mon[25866]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:27.120 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T08:19:28.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:27 smithi047 ceph-mon[25866]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:28.346 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:28 smithi203 systemd[1]: Starting Ceph mon.c for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:19:29.170 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 podman[30176]: 2024-09-17 08:19:28.971198146 +0000 UTC m=+0.095767350 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:19:29.171 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 podman[30176]: 2024-09-17 08:19:29.170745978 +0000 UTC m=+0.295315177 container create 66cd9b73a652108bffa0e6361733a90223127ea43bcd309d18368fdf1848d995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-c, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-17T08:19:29.290 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-17T08:19:29.290 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"3a66518a-74cd-11ef-bceb-c7b262605968","modified":"2024-09-17T08:17:53.722837Z","created":"2024-09-17T08:17:53.722837Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-17T08:19:29.290 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-09-17T08:19:29.421 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 podman[30176]: 2024-09-17 08:19:29.395226431 +0000 UTC m=+0.519795646 container init 66cd9b73a652108bffa0e6361733a90223127ea43bcd309d18368fdf1848d995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-17T08:19:29.422 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 podman[30176]: 2024-09-17 08:19:29.40032799 +0000 UTC m=+0.524897208 container start 66cd9b73a652108bffa0e6361733a90223127ea43bcd309d18368fdf1848d995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-17T08:19:29.736 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: set uid:gid to 167:167 (ceph:ceph) 2024-09-17T08:19:29.736 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable), process ceph-mon, pid 7 2024-09-17T08:19:29.737 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: pidfile_write: ignore empty --pid-file 2024-09-17T08:19:29.737 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: load: jerasure load: lrc 2024-09-17T08:19:29.737 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: RocksDB version: 7.9.2 2024-09-17T08:19:29.737 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Git sha 0 2024-09-17T08:19:29.737 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Compile date 2024-09-16 15:15:13 2024-09-17T08:19:29.737 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: DB SUMMARY 2024-09-17T08:19:29.737 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: DB Session ID: BKYLBUNCCAYJ570A0MGZ 2024-09-17T08:19:29.737 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: CURRENT file: CURRENT 2024-09-17T08:19:29.737 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: IDENTITY file: IDENTITY 2024-09-17T08:19:29.737 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-17T08:19:29.737 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-17T08:19:29.738 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-17T08:19:29.738 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.error_if_exists: 0 2024-09-17T08:19:29.738 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.create_if_missing: 0 2024-09-17T08:19:29.738 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.paranoid_checks: 1 2024-09-17T08:19:29.738 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-17T08:19:29.738 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-17T08:19:29.738 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-17T08:19:29.738 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.env: 0x555a095f7bc0 2024-09-17T08:19:29.738 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.fs: PosixFileSystem 2024-09-17T08:19:29.738 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.info_log: 0x555a0bda8be0 2024-09-17T08:19:29.738 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_file_opening_threads: 16 2024-09-17T08:19:29.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.statistics: (nil) 2024-09-17T08:19:29.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.use_fsync: 0 2024-09-17T08:19:29.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_log_file_size: 0 2024-09-17T08:19:29.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-17T08:19:29.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-17T08:19:29.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.keep_log_file_num: 1000 2024-09-17T08:19:29.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.recycle_log_file_num: 0 2024-09-17T08:19:29.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.allow_fallocate: 1 2024-09-17T08:19:29.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.allow_mmap_reads: 0 2024-09-17T08:19:29.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.allow_mmap_writes: 0 2024-09-17T08:19:29.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.use_direct_reads: 0 2024-09-17T08:19:29.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-17T08:19:29.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.create_missing_column_families: 0 2024-09-17T08:19:29.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.db_log_dir: 2024-09-17T08:19:29.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.wal_dir: 2024-09-17T08:19:29.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-17T08:19:29.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-17T08:19:29.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-17T08:19:29.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-17T08:19:29.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-17T08:19:29.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-17T08:19:29.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.advise_random_on_open: 1 2024-09-17T08:19:29.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.db_write_buffer_size: 0 2024-09-17T08:19:29.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.write_buffer_manager: 0x555a0bdb6320 2024-09-17T08:19:29.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-17T08:19:29.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-17T08:19:29.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-17T08:19:29.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.rate_limiter: (nil) 2024-09-17T08:19:29.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-17T08:19:29.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.wal_recovery_mode: 2 2024-09-17T08:19:29.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.enable_thread_tracking: 0 2024-09-17T08:19:29.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.enable_pipelined_write: 0 2024-09-17T08:19:29.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.unordered_write: 0 2024-09-17T08:19:29.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-17T08:19:29.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-17T08:19:29.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-17T08:19:29.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-17T08:19:29.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.row_cache: None 2024-09-17T08:19:29.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.wal_filter: None 2024-09-17T08:19:29.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-17T08:19:29.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.allow_ingest_behind: 0 2024-09-17T08:19:29.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.two_write_queues: 0 2024-09-17T08:19:29.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.manual_wal_flush: 0 2024-09-17T08:19:29.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.wal_compression: 0 2024-09-17T08:19:29.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.atomic_flush: 0 2024-09-17T08:19:29.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-17T08:19:29.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-17T08:19:29.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-17T08:19:29.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.log_readahead_size: 0 2024-09-17T08:19:29.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-17T08:19:29.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.best_efforts_recovery: 0 2024-09-17T08:19:29.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-17T08:19:29.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-17T08:19:29.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.allow_data_in_errors: 0 2024-09-17T08:19:29.743 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.db_host_id: __hostname__ 2024-09-17T08:19:29.743 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-17T08:19:29.744 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_background_jobs: 2 2024-09-17T08:19:29.744 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_background_compactions: -1 2024-09-17T08:19:29.744 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_subcompactions: 1 2024-09-17T08:19:29.744 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-17T08:19:29.744 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-17T08:19:29.744 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-17T08:19:29.744 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_total_wal_size: 0 2024-09-17T08:19:29.744 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-17T08:19:29.744 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-17T08:19:29.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-17T08:19:29.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-17T08:19:29.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_open_files: -1 2024-09-17T08:19:29.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.bytes_per_sync: 0 2024-09-17T08:19:29.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-17T08:19:29.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-17T08:19:29.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compaction_readahead_size: 0 2024-09-17T08:19:29.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_background_flushes: -1 2024-09-17T08:19:29.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Compression algorithms supported: 2024-09-17T08:19:29.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: kZSTD supported: 0 2024-09-17T08:19:29.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: kXpressCompression supported: 0 2024-09-17T08:19:29.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: kBZip2Compression supported: 0 2024-09-17T08:19:29.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-17T08:19:29.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: kLZ4Compression supported: 1 2024-09-17T08:19:29.745 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: kZlibCompression supported: 1 2024-09-17T08:19:29.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: kLZ4HCCompression supported: 1 2024-09-17T08:19:29.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: kSnappyCompression supported: 1 2024-09-17T08:19:29.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-17T08:19:29.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-17T08:19:29.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-17T08:19:29.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-17T08:19:29.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-17T08:19:29.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.merge_operator: 2024-09-17T08:19:29.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compaction_filter: None 2024-09-17T08:19:29.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compaction_filter_factory: None 2024-09-17T08:19:29.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.sst_partitioner_factory: None 2024-09-17T08:19:29.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-17T08:19:29.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-17T08:19:29.746 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555a0bda87e0) 2024-09-17T08:19:29.747 INFO:journalctl@ceph.mon.c.smithi203.stdout: cache_index_and_filter_blocks: 1 2024-09-17T08:19:29.747 INFO:journalctl@ceph.mon.c.smithi203.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-17T08:19:29.747 INFO:journalctl@ceph.mon.c.smithi203.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-17T08:19:29.747 INFO:journalctl@ceph.mon.c.smithi203.stdout: pin_top_level_index_and_filter: 1 2024-09-17T08:19:29.747 INFO:journalctl@ceph.mon.c.smithi203.stdout: index_type: 0 2024-09-17T08:19:29.747 INFO:journalctl@ceph.mon.c.smithi203.stdout: data_block_index_type: 0 2024-09-17T08:19:29.747 INFO:journalctl@ceph.mon.c.smithi203.stdout: index_shortening: 1 2024-09-17T08:19:29.747 INFO:journalctl@ceph.mon.c.smithi203.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-17T08:19:29.747 INFO:journalctl@ceph.mon.c.smithi203.stdout: checksum: 4 2024-09-17T08:19:29.747 INFO:journalctl@ceph.mon.c.smithi203.stdout: no_block_cache: 0 2024-09-17T08:19:29.747 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_cache: 0x555a0bd9d350 2024-09-17T08:19:29.747 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_cache_name: BinnedLRUCache 2024-09-17T08:19:29.747 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_cache_options: 2024-09-17T08:19:29.748 INFO:journalctl@ceph.mon.c.smithi203.stdout: capacity : 536870912 2024-09-17T08:19:29.748 INFO:journalctl@ceph.mon.c.smithi203.stdout: num_shard_bits : 4 2024-09-17T08:19:29.748 INFO:journalctl@ceph.mon.c.smithi203.stdout: strict_capacity_limit : 0 2024-09-17T08:19:29.748 INFO:journalctl@ceph.mon.c.smithi203.stdout: high_pri_pool_ratio: 0.000 2024-09-17T08:19:29.748 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_cache_compressed: (nil) 2024-09-17T08:19:29.748 INFO:journalctl@ceph.mon.c.smithi203.stdout: persistent_cache: (nil) 2024-09-17T08:19:29.748 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_size: 4096 2024-09-17T08:19:29.748 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_size_deviation: 10 2024-09-17T08:19:29.748 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_restart_interval: 16 2024-09-17T08:19:29.748 INFO:journalctl@ceph.mon.c.smithi203.stdout: index_block_restart_interval: 1 2024-09-17T08:19:29.748 INFO:journalctl@ceph.mon.c.smithi203.stdout: metadata_block_size: 4096 2024-09-17T08:19:29.748 INFO:journalctl@ceph.mon.c.smithi203.stdout: partition_filters: 0 2024-09-17T08:19:29.749 INFO:journalctl@ceph.mon.c.smithi203.stdout: use_delta_encoding: 1 2024-09-17T08:19:29.749 INFO:journalctl@ceph.mon.c.smithi203.stdout: filter_policy: bloomfilter 2024-09-17T08:19:29.749 INFO:journalctl@ceph.mon.c.smithi203.stdout: whole_key_filtering: 1 2024-09-17T08:19:29.749 INFO:journalctl@ceph.mon.c.smithi203.stdout: verify_compression: 0 2024-09-17T08:19:29.749 INFO:journalctl@ceph.mon.c.smithi203.stdout: read_amp_bytes_per_bit: 0 2024-09-17T08:19:29.749 INFO:journalctl@ceph.mon.c.smithi203.stdout: format_version: 5 2024-09-17T08:19:29.749 INFO:journalctl@ceph.mon.c.smithi203.stdout: enable_index_compression: 1 2024-09-17T08:19:29.749 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_align: 0 2024-09-17T08:19:29.749 INFO:journalctl@ceph.mon.c.smithi203.stdout: max_auto_readahead_size: 262144 2024-09-17T08:19:29.749 INFO:journalctl@ceph.mon.c.smithi203.stdout: prepopulate_block_cache: 0 2024-09-17T08:19:29.749 INFO:journalctl@ceph.mon.c.smithi203.stdout: initial_auto_readahead_size: 8192 2024-09-17T08:19:29.749 INFO:journalctl@ceph.mon.c.smithi203.stdout: num_file_reads_for_auto_readahead: 2 2024-09-17T08:19:29.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.write_buffer_size: 33554432 2024-09-17T08:19:29.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_write_buffer_number: 2 2024-09-17T08:19:29.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compression: NoCompression 2024-09-17T08:19:29.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.bottommost_compression: Disabled 2024-09-17T08:19:29.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.prefix_extractor: nullptr 2024-09-17T08:19:29.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-17T08:19:29.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.num_levels: 7 2024-09-17T08:19:29.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-17T08:19:29.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-17T08:19:29.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-17T08:19:29.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-17T08:19:29.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-17T08:19:29.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-17T08:19:29.751 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-17T08:19:29.751 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-17T08:19:29.751 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-17T08:19:29.751 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-17T08:19:29.751 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-17T08:19:29.751 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-17T08:19:29.751 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-17T08:19:29.751 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compression_opts.level: 32767 2024-09-17T08:19:29.751 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compression_opts.strategy: 0 2024-09-17T08:19:29.751 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-17T08:19:29.751 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-17T08:19:29.751 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-17T08:19:29.752 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-17T08:19:29.752 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compression_opts.enabled: false 2024-09-17T08:19:29.752 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-17T08:19:29.752 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-17T08:19:29.752 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-17T08:19:29.752 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-17T08:19:29.752 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.target_file_size_base: 67108864 2024-09-17T08:19:29.752 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-17T08:19:29.752 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-17T08:19:29.752 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-17T08:19:29.753 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-17T08:19:29.753 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-17T08:19:29.753 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-17T08:19:29.753 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-17T08:19:29.753 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-17T08:19:29.753 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-17T08:19:29.753 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-17T08:19:29.753 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-17T08:19:29.753 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-17T08:19:29.753 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-17T08:19:29.753 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-17T08:19:29.753 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.arena_block_size: 1048576 2024-09-17T08:19:29.754 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-17T08:19:29.754 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-17T08:19:29.754 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.disable_auto_compactions: 0 2024-09-17T08:19:29.754 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-17T08:19:29.754 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-17T08:19:29.754 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-17T08:19:29.754 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-17T08:19:29.754 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-17T08:19:29.754 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-17T08:19:29.754 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-17T08:19:29.754 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-17T08:19:29.755 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-17T08:19:29.755 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-17T08:19:29.755 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-17T08:19:29.755 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.inplace_update_support: 0 2024-09-17T08:19:29.755 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-17T08:19:29.755 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-17T08:19:29.755 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-17T08:19:29.755 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-17T08:19:29.755 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.bloom_locality: 0 2024-09-17T08:19:29.755 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.max_successive_merges: 0 2024-09-17T08:19:29.755 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-17T08:19:29.755 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.paranoid_file_checks: 0 2024-09-17T08:19:29.756 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.force_consistency_checks: 1 2024-09-17T08:19:29.756 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.report_bg_io_stats: 0 2024-09-17T08:19:29.756 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.ttl: 2592000 2024-09-17T08:19:29.756 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-17T08:19:29.756 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-17T08:19:29.756 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-17T08:19:29.756 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.enable_blob_files: false 2024-09-17T08:19:29.756 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.min_blob_size: 0 2024-09-17T08:19:29.756 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.blob_file_size: 268435456 2024-09-17T08:19:29.756 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-17T08:19:29.756 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-17T08:19:29.756 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-17T08:19:29.756 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-17T08:19:29.757 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-17T08:19:29.757 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.blob_file_starting_level: 0 2024-09-17T08:19:29.757 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-17T08:19:29.757 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-17T08:19:29.757 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-17T08:19:29.757 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e9f1ad72-23bc-4797-821b-8b209489e494 2024-09-17T08:19:29.757 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726561169427809, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-17T08:19:29.757 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-17T08:19:29.757 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726561169428228, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726561169, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e9f1ad72-23bc-4797-821b-8b209489e494", "db_session_id": "BKYLBUNCCAYJ570A0MGZ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-17T08:19:29.757 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726561169428338, "job": 1, "event": "recovery_finished"} 2024-09-17T08:19:29.757 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-17T08:19:29.757 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-17T08:19:29.757 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555a0bdc8e00 2024-09-17T08:19:29.758 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: DB pointer 0x555a0bebc000 2024-09-17T08:19:29.758 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-17T08:19:29.758 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-17T08:19:29.758 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** DB Stats ** 2024-09-17T08:19:29.758 INFO:journalctl@ceph.mon.c.smithi203.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T08:19:29.758 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-17T08:19:29.758 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T08:19:29.758 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T08:19:29.758 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-17T08:19:29.758 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T08:19:29.758 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T08:19:29.758 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-09-17T08:19:29.758 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** Compaction Stats [default] ** 2024-09-17T08:19:29.758 INFO:journalctl@ceph.mon.c.smithi203.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T08:19:29.759 INFO:journalctl@ceph.mon.c.smithi203.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-17T08:19:29.759 INFO:journalctl@ceph.mon.c.smithi203.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:19:29.759 INFO:journalctl@ceph.mon.c.smithi203.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:19:29.759 INFO:journalctl@ceph.mon.c.smithi203.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:19:29.759 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-09-17T08:19:29.759 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** Compaction Stats [default] ** 2024-09-17T08:19:29.759 INFO:journalctl@ceph.mon.c.smithi203.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T08:19:29.759 INFO:journalctl@ceph.mon.c.smithi203.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-17T08:19:29.759 INFO:journalctl@ceph.mon.c.smithi203.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:19:29.759 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-09-17T08:19:29.759 INFO:journalctl@ceph.mon.c.smithi203.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-17T08:19:29.759 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-09-17T08:19:29.759 INFO:journalctl@ceph.mon.c.smithi203.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T08:19:29.760 INFO:journalctl@ceph.mon.c.smithi203.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-17T08:19:29.760 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-17T08:19:29.760 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-17T08:19:29.760 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-17T08:19:29.760 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-17T08:19:29.760 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T08:19:29.760 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T08:19:29.760 INFO:journalctl@ceph.mon.c.smithi203.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-17T08:19:29.760 INFO:journalctl@ceph.mon.c.smithi203.stdout: Block cache BinnedLRUCache@0x555a0bd9d350#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-09-17T08:19:29.760 INFO:journalctl@ceph.mon.c.smithi203.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-17T08:19:29.760 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-09-17T08:19:29.760 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-17T08:19:29.760 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-17T08:19:29.761 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: using public_addr v2:172.21.15.203:0/0 -> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] 2024-09-17T08:19:29.761 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: starting mon.c rank -1 at public addrs [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] at bind addrs [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:19:29.761 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: mon.c@-1(???) e0 preinit fsid 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:19:29.761 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: mon.c@-1(synchronizing).mds e1 new map 2024-09-17T08:19:29.761 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-17T08:19:29.761 INFO:journalctl@ceph.mon.c.smithi203.stdout: e1 2024-09-17T08:19:29.761 INFO:journalctl@ceph.mon.c.smithi203.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T08:19:29.761 INFO:journalctl@ceph.mon.c.smithi203.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T08:19:29.761 INFO:journalctl@ceph.mon.c.smithi203.stdout: legacy client fscid: -1 2024-09-17T08:19:29.761 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-09-17T08:19:29.761 INFO:journalctl@ceph.mon.c.smithi203.stdout: No filesystems configured 2024-09-17T08:19:29.761 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-17T08:19:29.761 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-17T08:19:29.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-17T08:19:29.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-17T08:19:29.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-17T08:19:29.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-17T08:19:29.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-17T08:19:29.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:19:29.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:19:29.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:19:29.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:19:29.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: Updating smithi193:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:19:29.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: Deploying cephadm binary to smithi203 2024-09-17T08:19:29.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:29.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:29.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: Added host smithi203 2024-09-17T08:19:29.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:29.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:29.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:29.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:29.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:29.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:19:29.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:29.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:29.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:29.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:29.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:29.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:19:29.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:29.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:19:29.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-17T08:19:29.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: Updating smithi203:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:19:29.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:19:29.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:29.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/1030672606' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-17T08:19:29.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:29.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:29.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:29.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: Updating smithi203:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:19:29.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/1030672606' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-17T08:19:29.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:19:29.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:29.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi193:172.21.15.193=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:19:29.765 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: Saving service mon spec with placement smithi047:172.21.15.47=a;smithi193:172.21.15.193=b;smithi203:172.21.15.203=c;count:3 2024-09-17T08:19:29.765 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:29.765 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:29.765 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:29.765 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:19:29.765 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:29.765 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:19:29.765 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:29.765 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: Deploying daemon mon.c on smithi203 2024-09-17T08:19:29.765 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:29.765 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:29.765 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: from='client.? 172.21.15.203:0/3574688689' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:19:29.765 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 ceph-mon[30259]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-17T08:19:29.765 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 bash[30176]: 66cd9b73a652108bffa0e6361733a90223127ea43bcd309d18368fdf1848d995 2024-09-17T08:19:29.766 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:29 smithi203 systemd[1]: Started Ceph mon.c for 3a66518a-74cd-11ef-bceb-c7b262605968. 2024-09-17T08:19:29.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:29 smithi047 ceph-mon[25866]: from='client.? 172.21.15.203:0/3574688689' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:19:31.112 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-17T08:19:31.113 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-17T08:19:31.319 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T08:19:34.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:19:34.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: mon.a calling monitor election 2024-09-17T08:19:34.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:34.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:34.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:34.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:34.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: mon.c calling monitor election 2024-09-17T08:19:34.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:34.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:34.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:34.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:34.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:34.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-17T08:19:34.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: monmap e2: 2 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],c=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:19:34.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: fsmap 2024-09-17T08:19:34.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:19:34.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: mgrmap e13: y(active, since 46s) 2024-09-17T08:19:34.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: overall HEALTH_OK 2024-09-17T08:19:34.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:34.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:34.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:34.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:19:34.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:34 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:34.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:19:34.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: mon.a calling monitor election 2024-09-17T08:19:34.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:34.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:34.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:34.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:34.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: mon.c calling monitor election 2024-09-17T08:19:34.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:34.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:34.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:34.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-17T08:19:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: monmap e2: 2 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],c=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:19:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: fsmap 2024-09-17T08:19:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:19:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: mgrmap e13: y(active, since 46s) 2024-09-17T08:19:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: overall HEALTH_OK 2024-09-17T08:19:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:19:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:34 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:35.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:35 smithi203 ceph-mon[30259]: Deploying daemon mon.b on smithi193 2024-09-17T08:19:35.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:35 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:35.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:35 smithi047 ceph-mon[25866]: Deploying daemon mon.b on smithi193 2024-09-17T08:19:35.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:35 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:36.013 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-17T08:19:36.013 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":2,"fsid":"3a66518a-74cd-11ef-bceb-c7b262605968","modified":"2024-09-17T08:19:29.455627Z","created":"2024-09-17T08:17:53.722837Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-17T08:19:36.013 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 2 2024-09-17T08:19:36.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:36 smithi203 ceph-mon[30259]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:36.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:36 smithi203 ceph-mon[30259]: from='client.? 172.21.15.203:0/2760449646' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:19:36.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:36 smithi047 ceph-mon[25866]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:36.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:36 smithi047 ceph-mon[25866]: from='client.? 172.21.15.203:0/2760449646' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:19:37.517 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-17T08:19:37.518 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-17T08:19:37.718 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T08:19:37.838 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:37 smithi203 ceph-mon[30259]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:37.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:37 smithi193 podman[29964]: 2024-09-17 08:19:37.797117042 +0000 UTC m=+0.132250790 container create cc6a76045055bd58b90ab0b1a0dfe3bab17bb8ce1f851ef58483152306fac624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-17T08:19:38.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:37 smithi047 ceph-mon[25866]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:38.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:37 smithi193 podman[29964]: 2024-09-17 08:19:37.982749266 +0000 UTC m=+0.317883024 container init cc6a76045055bd58b90ab0b1a0dfe3bab17bb8ce1f851ef58483152306fac624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-b, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-17T08:19:38.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:37 smithi193 podman[29964]: 2024-09-17 08:19:37.987493861 +0000 UTC m=+0.322627610 container start cc6a76045055bd58b90ab0b1a0dfe3bab17bb8ce1f851ef58483152306fac624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-17T08:19:38.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: set uid:gid to 167:167 (ceph:ceph) 2024-09-17T08:19:38.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable), process ceph-mon, pid 7 2024-09-17T08:19:38.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: pidfile_write: ignore empty --pid-file 2024-09-17T08:19:38.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: load: jerasure load: lrc 2024-09-17T08:19:38.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: RocksDB version: 7.9.2 2024-09-17T08:19:38.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Git sha 0 2024-09-17T08:19:38.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Compile date 2024-09-16 15:15:13 2024-09-17T08:19:38.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: DB SUMMARY 2024-09-17T08:19:38.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: DB Session ID: H0SA9FEJGA1NC0D0GU4S 2024-09-17T08:19:38.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: CURRENT file: CURRENT 2024-09-17T08:19:38.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: IDENTITY file: IDENTITY 2024-09-17T08:19:38.396 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-17T08:19:38.396 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-17T08:19:38.396 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-17T08:19:38.396 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.error_if_exists: 0 2024-09-17T08:19:38.396 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.create_if_missing: 0 2024-09-17T08:19:38.396 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.paranoid_checks: 1 2024-09-17T08:19:38.396 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-17T08:19:38.396 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-17T08:19:38.396 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-17T08:19:38.396 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.env: 0x556bef773bc0 2024-09-17T08:19:38.397 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.fs: PosixFileSystem 2024-09-17T08:19:38.397 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.info_log: 0x556bf002abe0 2024-09-17T08:19:38.397 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_file_opening_threads: 16 2024-09-17T08:19:38.397 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.statistics: (nil) 2024-09-17T08:19:38.397 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.use_fsync: 0 2024-09-17T08:19:38.397 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_log_file_size: 0 2024-09-17T08:19:38.397 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-17T08:19:38.397 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-17T08:19:38.397 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.keep_log_file_num: 1000 2024-09-17T08:19:38.397 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.recycle_log_file_num: 0 2024-09-17T08:19:38.397 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.allow_fallocate: 1 2024-09-17T08:19:38.397 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.allow_mmap_reads: 0 2024-09-17T08:19:38.397 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.allow_mmap_writes: 0 2024-09-17T08:19:38.398 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.use_direct_reads: 0 2024-09-17T08:19:38.398 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-17T08:19:38.398 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.create_missing_column_families: 0 2024-09-17T08:19:38.398 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.db_log_dir: 2024-09-17T08:19:38.398 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.wal_dir: 2024-09-17T08:19:38.398 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-17T08:19:38.398 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-17T08:19:38.398 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-17T08:19:38.398 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-17T08:19:38.398 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-17T08:19:38.398 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-17T08:19:38.399 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.advise_random_on_open: 1 2024-09-17T08:19:38.399 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.db_write_buffer_size: 0 2024-09-17T08:19:38.399 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.write_buffer_manager: 0x556bf0038320 2024-09-17T08:19:38.399 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-17T08:19:38.399 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-17T08:19:38.399 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-17T08:19:38.399 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.rate_limiter: (nil) 2024-09-17T08:19:38.399 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-17T08:19:38.399 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.wal_recovery_mode: 2 2024-09-17T08:19:38.399 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.enable_thread_tracking: 0 2024-09-17T08:19:38.399 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.enable_pipelined_write: 0 2024-09-17T08:19:38.399 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.unordered_write: 0 2024-09-17T08:19:38.399 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-17T08:19:38.400 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-17T08:19:38.400 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-17T08:19:38.400 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-17T08:19:38.400 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.row_cache: None 2024-09-17T08:19:38.400 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.wal_filter: None 2024-09-17T08:19:38.401 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-17T08:19:38.401 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.allow_ingest_behind: 0 2024-09-17T08:19:38.402 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.two_write_queues: 0 2024-09-17T08:19:38.402 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.manual_wal_flush: 0 2024-09-17T08:19:38.402 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.wal_compression: 0 2024-09-17T08:19:38.402 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.atomic_flush: 0 2024-09-17T08:19:38.402 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-17T08:19:38.402 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-17T08:19:38.402 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-17T08:19:38.402 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.log_readahead_size: 0 2024-09-17T08:19:38.402 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-17T08:19:38.402 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.best_efforts_recovery: 0 2024-09-17T08:19:38.402 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-17T08:19:38.402 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-17T08:19:38.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.allow_data_in_errors: 0 2024-09-17T08:19:38.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.db_host_id: __hostname__ 2024-09-17T08:19:38.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-17T08:19:38.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_background_jobs: 2 2024-09-17T08:19:38.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_background_compactions: -1 2024-09-17T08:19:38.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_subcompactions: 1 2024-09-17T08:19:38.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-17T08:19:38.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-17T08:19:38.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-17T08:19:38.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_total_wal_size: 0 2024-09-17T08:19:38.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-17T08:19:38.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-17T08:19:38.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-17T08:19:38.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-17T08:19:38.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_open_files: -1 2024-09-17T08:19:38.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.bytes_per_sync: 0 2024-09-17T08:19:38.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-17T08:19:38.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-17T08:19:38.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compaction_readahead_size: 0 2024-09-17T08:19:38.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_background_flushes: -1 2024-09-17T08:19:38.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Compression algorithms supported: 2024-09-17T08:19:38.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: kZSTD supported: 0 2024-09-17T08:19:38.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: kXpressCompression supported: 0 2024-09-17T08:19:38.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: kBZip2Compression supported: 0 2024-09-17T08:19:38.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-17T08:19:38.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: kLZ4Compression supported: 1 2024-09-17T08:19:38.405 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: kZlibCompression supported: 1 2024-09-17T08:19:38.405 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: kLZ4HCCompression supported: 1 2024-09-17T08:19:38.405 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: kSnappyCompression supported: 1 2024-09-17T08:19:38.405 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-17T08:19:38.405 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-17T08:19:38.405 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-17T08:19:38.405 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-17T08:19:38.405 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-17T08:19:38.405 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.merge_operator: 2024-09-17T08:19:38.405 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compaction_filter: None 2024-09-17T08:19:38.405 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compaction_filter_factory: None 2024-09-17T08:19:38.405 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.sst_partitioner_factory: None 2024-09-17T08:19:38.405 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-17T08:19:38.406 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-17T08:19:38.406 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556bf002a7e0) 2024-09-17T08:19:38.406 INFO:journalctl@ceph.mon.b.smithi193.stdout: cache_index_and_filter_blocks: 1 2024-09-17T08:19:38.406 INFO:journalctl@ceph.mon.b.smithi193.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-17T08:19:38.406 INFO:journalctl@ceph.mon.b.smithi193.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-17T08:19:38.406 INFO:journalctl@ceph.mon.b.smithi193.stdout: pin_top_level_index_and_filter: 1 2024-09-17T08:19:38.406 INFO:journalctl@ceph.mon.b.smithi193.stdout: index_type: 0 2024-09-17T08:19:38.406 INFO:journalctl@ceph.mon.b.smithi193.stdout: data_block_index_type: 0 2024-09-17T08:19:38.406 INFO:journalctl@ceph.mon.b.smithi193.stdout: index_shortening: 1 2024-09-17T08:19:38.406 INFO:journalctl@ceph.mon.b.smithi193.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-17T08:19:38.406 INFO:journalctl@ceph.mon.b.smithi193.stdout: checksum: 4 2024-09-17T08:19:38.406 INFO:journalctl@ceph.mon.b.smithi193.stdout: no_block_cache: 0 2024-09-17T08:19:38.406 INFO:journalctl@ceph.mon.b.smithi193.stdout: block_cache: 0x556bf001f1f0 2024-09-17T08:19:38.407 INFO:journalctl@ceph.mon.b.smithi193.stdout: block_cache_name: BinnedLRUCache 2024-09-17T08:19:38.407 INFO:journalctl@ceph.mon.b.smithi193.stdout: block_cache_options: 2024-09-17T08:19:38.407 INFO:journalctl@ceph.mon.b.smithi193.stdout: capacity : 536870912 2024-09-17T08:19:38.407 INFO:journalctl@ceph.mon.b.smithi193.stdout: num_shard_bits : 4 2024-09-17T08:19:38.407 INFO:journalctl@ceph.mon.b.smithi193.stdout: strict_capacity_limit : 0 2024-09-17T08:19:38.407 INFO:journalctl@ceph.mon.b.smithi193.stdout: high_pri_pool_ratio: 0.000 2024-09-17T08:19:38.407 INFO:journalctl@ceph.mon.b.smithi193.stdout: block_cache_compressed: (nil) 2024-09-17T08:19:38.407 INFO:journalctl@ceph.mon.b.smithi193.stdout: persistent_cache: (nil) 2024-09-17T08:19:38.407 INFO:journalctl@ceph.mon.b.smithi193.stdout: block_size: 4096 2024-09-17T08:19:38.407 INFO:journalctl@ceph.mon.b.smithi193.stdout: block_size_deviation: 10 2024-09-17T08:19:38.407 INFO:journalctl@ceph.mon.b.smithi193.stdout: block_restart_interval: 16 2024-09-17T08:19:38.407 INFO:journalctl@ceph.mon.b.smithi193.stdout: index_block_restart_interval: 1 2024-09-17T08:19:38.407 INFO:journalctl@ceph.mon.b.smithi193.stdout: metadata_block_size: 4096 2024-09-17T08:19:38.407 INFO:journalctl@ceph.mon.b.smithi193.stdout: partition_filters: 0 2024-09-17T08:19:38.408 INFO:journalctl@ceph.mon.b.smithi193.stdout: use_delta_encoding: 1 2024-09-17T08:19:38.408 INFO:journalctl@ceph.mon.b.smithi193.stdout: filter_policy: bloomfilter 2024-09-17T08:19:38.408 INFO:journalctl@ceph.mon.b.smithi193.stdout: whole_key_filtering: 1 2024-09-17T08:19:38.408 INFO:journalctl@ceph.mon.b.smithi193.stdout: verify_compression: 0 2024-09-17T08:19:38.408 INFO:journalctl@ceph.mon.b.smithi193.stdout: read_amp_bytes_per_bit: 0 2024-09-17T08:19:38.408 INFO:journalctl@ceph.mon.b.smithi193.stdout: format_version: 5 2024-09-17T08:19:38.408 INFO:journalctl@ceph.mon.b.smithi193.stdout: enable_index_compression: 1 2024-09-17T08:19:38.408 INFO:journalctl@ceph.mon.b.smithi193.stdout: block_align: 0 2024-09-17T08:19:38.408 INFO:journalctl@ceph.mon.b.smithi193.stdout: max_auto_readahead_size: 262144 2024-09-17T08:19:38.408 INFO:journalctl@ceph.mon.b.smithi193.stdout: prepopulate_block_cache: 0 2024-09-17T08:19:38.408 INFO:journalctl@ceph.mon.b.smithi193.stdout: initial_auto_readahead_size: 8192 2024-09-17T08:19:38.408 INFO:journalctl@ceph.mon.b.smithi193.stdout: num_file_reads_for_auto_readahead: 2 2024-09-17T08:19:38.408 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.write_buffer_size: 33554432 2024-09-17T08:19:38.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_write_buffer_number: 2 2024-09-17T08:19:38.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compression: NoCompression 2024-09-17T08:19:38.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.bottommost_compression: Disabled 2024-09-17T08:19:38.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.prefix_extractor: nullptr 2024-09-17T08:19:38.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-17T08:19:38.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.num_levels: 7 2024-09-17T08:19:38.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-17T08:19:38.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-17T08:19:38.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-17T08:19:38.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-17T08:19:38.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-17T08:19:38.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-17T08:19:38.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-17T08:19:38.410 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-17T08:19:38.410 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-17T08:19:38.410 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-17T08:19:38.410 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-17T08:19:38.410 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-17T08:19:38.410 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-17T08:19:38.410 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compression_opts.level: 32767 2024-09-17T08:19:38.410 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compression_opts.strategy: 0 2024-09-17T08:19:38.410 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-17T08:19:38.410 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-17T08:19:38.410 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-17T08:19:38.410 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-17T08:19:38.410 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compression_opts.enabled: false 2024-09-17T08:19:38.411 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-17T08:19:38.411 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-17T08:19:38.411 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-17T08:19:38.411 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-17T08:19:38.411 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.target_file_size_base: 67108864 2024-09-17T08:19:38.411 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-17T08:19:38.411 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-17T08:19:38.411 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-17T08:19:38.411 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-17T08:19:38.411 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-17T08:19:38.411 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-17T08:19:38.411 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-17T08:19:38.411 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-17T08:19:38.412 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-17T08:19:38.412 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-17T08:19:38.412 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-17T08:19:38.412 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-17T08:19:38.412 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-17T08:19:38.412 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-17T08:19:38.412 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.arena_block_size: 1048576 2024-09-17T08:19:38.412 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-17T08:19:38.412 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-17T08:19:38.412 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.disable_auto_compactions: 0 2024-09-17T08:19:38.412 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-17T08:19:38.412 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-17T08:19:38.412 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-17T08:19:38.412 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-17T08:19:38.413 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-17T08:19:38.413 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-17T08:19:38.413 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-17T08:19:38.413 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-17T08:19:38.413 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-17T08:19:38.413 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-17T08:19:38.413 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-17T08:19:38.413 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.inplace_update_support: 0 2024-09-17T08:19:38.413 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-17T08:19:38.413 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-17T08:19:38.413 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-17T08:19:38.413 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-17T08:19:38.413 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.bloom_locality: 0 2024-09-17T08:19:38.414 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.max_successive_merges: 0 2024-09-17T08:19:38.414 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-17T08:19:38.414 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.paranoid_file_checks: 0 2024-09-17T08:19:38.414 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.force_consistency_checks: 1 2024-09-17T08:19:38.414 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.report_bg_io_stats: 0 2024-09-17T08:19:38.414 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.ttl: 2592000 2024-09-17T08:19:38.414 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-17T08:19:38.414 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-17T08:19:38.414 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-17T08:19:38.414 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.enable_blob_files: false 2024-09-17T08:19:38.414 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.min_blob_size: 0 2024-09-17T08:19:38.414 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.blob_file_size: 268435456 2024-09-17T08:19:38.414 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-17T08:19:38.414 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-17T08:19:38.415 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-17T08:19:38.415 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-17T08:19:38.415 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-17T08:19:38.415 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.blob_file_starting_level: 0 2024-09-17T08:19:38.415 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-17T08:19:38.415 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-17T08:19:38.415 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-17T08:19:38.415 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 27571cd5-a26e-42c1-92a3-93ca57092e16 2024-09-17T08:19:38.415 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726561178014323, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-17T08:19:38.415 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-17T08:19:38.415 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726561178014721, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726561178, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "27571cd5-a26e-42c1-92a3-93ca57092e16", "db_session_id": "H0SA9FEJGA1NC0D0GU4S", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-17T08:19:38.415 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726561178014813, "job": 1, "event": "recovery_finished"} 2024-09-17T08:19:38.416 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-17T08:19:38.416 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-17T08:19:38.416 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556bf004ae00 2024-09-17T08:19:38.416 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: DB pointer 0x556bf013e000 2024-09-17T08:19:38.416 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-17T08:19:38.416 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-17T08:19:38.416 INFO:journalctl@ceph.mon.b.smithi193.stdout: ** DB Stats ** 2024-09-17T08:19:38.416 INFO:journalctl@ceph.mon.b.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T08:19:38.416 INFO:journalctl@ceph.mon.b.smithi193.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-17T08:19:38.416 INFO:journalctl@ceph.mon.b.smithi193.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T08:19:38.416 INFO:journalctl@ceph.mon.b.smithi193.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T08:19:38.416 INFO:journalctl@ceph.mon.b.smithi193.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-17T08:19:38.416 INFO:journalctl@ceph.mon.b.smithi193.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T08:19:38.416 INFO:journalctl@ceph.mon.b.smithi193.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T08:19:38.417 INFO:journalctl@ceph.mon.b.smithi193.stdout: 2024-09-17T08:19:38.417 INFO:journalctl@ceph.mon.b.smithi193.stdout: ** Compaction Stats [default] ** 2024-09-17T08:19:38.417 INFO:journalctl@ceph.mon.b.smithi193.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T08:19:38.417 INFO:journalctl@ceph.mon.b.smithi193.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-17T08:19:38.417 INFO:journalctl@ceph.mon.b.smithi193.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:19:38.417 INFO:journalctl@ceph.mon.b.smithi193.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:19:38.417 INFO:journalctl@ceph.mon.b.smithi193.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:19:38.417 INFO:journalctl@ceph.mon.b.smithi193.stdout: 2024-09-17T08:19:38.417 INFO:journalctl@ceph.mon.b.smithi193.stdout: ** Compaction Stats [default] ** 2024-09-17T08:19:38.417 INFO:journalctl@ceph.mon.b.smithi193.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T08:19:38.417 INFO:journalctl@ceph.mon.b.smithi193.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-17T08:19:38.417 INFO:journalctl@ceph.mon.b.smithi193.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:19:38.417 INFO:journalctl@ceph.mon.b.smithi193.stdout: 2024-09-17T08:19:38.418 INFO:journalctl@ceph.mon.b.smithi193.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-17T08:19:38.418 INFO:journalctl@ceph.mon.b.smithi193.stdout: 2024-09-17T08:19:38.418 INFO:journalctl@ceph.mon.b.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T08:19:38.418 INFO:journalctl@ceph.mon.b.smithi193.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-17T08:19:38.418 INFO:journalctl@ceph.mon.b.smithi193.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-17T08:19:38.418 INFO:journalctl@ceph.mon.b.smithi193.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-17T08:19:38.418 INFO:journalctl@ceph.mon.b.smithi193.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-17T08:19:38.418 INFO:journalctl@ceph.mon.b.smithi193.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-17T08:19:38.418 INFO:journalctl@ceph.mon.b.smithi193.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T08:19:38.419 INFO:journalctl@ceph.mon.b.smithi193.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T08:19:38.419 INFO:journalctl@ceph.mon.b.smithi193.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-17T08:19:38.419 INFO:journalctl@ceph.mon.b.smithi193.stdout: Block cache BinnedLRUCache@0x556bf001f1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-17T08:19:38.419 INFO:journalctl@ceph.mon.b.smithi193.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-17T08:19:38.419 INFO:journalctl@ceph.mon.b.smithi193.stdout: 2024-09-17T08:19:38.419 INFO:journalctl@ceph.mon.b.smithi193.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-17T08:19:38.419 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-17T08:19:38.419 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: using public_addr v2:172.21.15.193:0/0 -> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-09-17T08:19:38.419 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: starting mon.b rank -1 at public addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] at bind addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:19:38.419 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: mon.b@-1(???) e0 preinit fsid 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:19:38.419 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: mon.b@-1(synchronizing).mds e1 new map 2024-09-17T08:19:38.419 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-17T08:19:38.419 INFO:journalctl@ceph.mon.b.smithi193.stdout: e1 2024-09-17T08:19:38.419 INFO:journalctl@ceph.mon.b.smithi193.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T08:19:38.420 INFO:journalctl@ceph.mon.b.smithi193.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T08:19:38.420 INFO:journalctl@ceph.mon.b.smithi193.stdout: legacy client fscid: -1 2024-09-17T08:19:38.420 INFO:journalctl@ceph.mon.b.smithi193.stdout: 2024-09-17T08:19:38.420 INFO:journalctl@ceph.mon.b.smithi193.stdout: No filesystems configured 2024-09-17T08:19:38.420 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-17T08:19:38.420 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-17T08:19:38.420 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-17T08:19:38.420 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-17T08:19:38.420 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-17T08:19:38.420 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-17T08:19:38.420 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-17T08:19:38.420 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:19:38.420 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:19:38.420 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:19:38.421 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:19:38.421 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: Updating smithi193:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:19:38.421 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: Deploying cephadm binary to smithi203 2024-09-17T08:19:38.421 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:38.421 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:38.421 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: Added host smithi203 2024-09-17T08:19:38.421 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:38.421 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:38.421 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:38.421 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:38.421 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:38.421 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:19:38.421 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:38.421 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:38.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:38.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:38.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:38.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:19:38.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:38.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:19:38.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-17T08:19:38.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: Updating smithi203:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:19:38.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:19:38.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:38.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/1030672606' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-17T08:19:38.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:38.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:38.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:38.423 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: Updating smithi203:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:19:38.423 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/1030672606' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-17T08:19:38.423 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:19:38.423 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:38.423 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi193:172.21.15.193=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:19:38.423 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: Saving service mon spec with placement smithi047:172.21.15.47=a;smithi193:172.21.15.193=b;smithi203:172.21.15.203=c;count:3 2024-09-17T08:19:38.423 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:38.423 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:38.423 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:38.423 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:19:38.423 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:38.423 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:19:38.423 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:38.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: Deploying daemon mon.c on smithi203 2024-09-17T08:19:38.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:38.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:38.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='client.? 172.21.15.203:0/3574688689' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:19:38.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:19:38.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: mon.a calling monitor election 2024-09-17T08:19:38.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:38.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:38.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:38.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:38.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: mon.c calling monitor election 2024-09-17T08:19:38.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:38.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:38.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:38.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:38.425 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:38.425 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-17T08:19:38.425 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: monmap e2: 2 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],c=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:19:38.425 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: fsmap 2024-09-17T08:19:38.425 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:19:38.425 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: mgrmap e13: y(active, since 46s) 2024-09-17T08:19:38.425 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: overall HEALTH_OK 2024-09-17T08:19:38.425 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:38.425 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:38.425 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:38.425 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:19:38.425 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:38.425 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: Deploying daemon mon.b on smithi193 2024-09-17T08:19:38.425 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:38.425 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:38.426 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: from='client.? 172.21.15.203:0/2760449646' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:19:38.426 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:38.426 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 ceph-mon[30002]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-17T08:19:38.426 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 bash[29964]: cc6a76045055bd58b90ab0b1a0dfe3bab17bb8ce1f851ef58483152306fac624 2024-09-17T08:19:38.426 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:38 smithi193 systemd[1]: Started Ceph mon.b for 3a66518a-74cd-11ef-bceb-c7b262605968. 2024-09-17T08:19:43.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:19:43.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: mon.a calling monitor election 2024-09-17T08:19:43.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: mon.c calling monitor election 2024-09-17T08:19:43.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:19:43.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:43.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:19:43.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:43.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:19:43.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: mon.b calling monitor election 2024-09-17T08:19:43.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:19:43.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:43.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:19:43.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:19:43.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-17T08:19:43.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: monmap e3: 3 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0],c=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:19:43.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: fsmap 2024-09-17T08:19:43.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:19:43.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: mgrmap e13: y(active, since 55s) 2024-09-17T08:19:43.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: overall HEALTH_OK 2024-09-17T08:19:43.396 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:43.396 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:43.396 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:43.396 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:43 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:43.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:19:43.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: mon.a calling monitor election 2024-09-17T08:19:43.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: mon.c calling monitor election 2024-09-17T08:19:43.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:19:43.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:43.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:19:43.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:43.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:19:43.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: mon.b calling monitor election 2024-09-17T08:19:43.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:19:43.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:43.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:19:43.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:19:43.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-17T08:19:43.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: monmap e3: 3 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0],c=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:19:43.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: fsmap 2024-09-17T08:19:43.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:19:43.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: mgrmap e13: y(active, since 55s) 2024-09-17T08:19:43.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: overall HEALTH_OK 2024-09-17T08:19:43.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:43.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:43.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:43.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:43 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:43.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:19:43.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: mon.a calling monitor election 2024-09-17T08:19:43.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: mon.c calling monitor election 2024-09-17T08:19:43.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:19:43.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:19:43.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:19:43.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:43.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:19:43.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: mon.b calling monitor election 2024-09-17T08:19:43.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:19:43.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:43.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:19:43.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:19:43.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-17T08:19:43.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: monmap e3: 3 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0],c=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:19:43.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: fsmap 2024-09-17T08:19:43.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:19:43.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: mgrmap e13: y(active, since 55s) 2024-09-17T08:19:43.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: overall HEALTH_OK 2024-09-17T08:19:43.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:43.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:43.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:43.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:43 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:43.822 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-17T08:19:43.822 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":3,"fsid":"3a66518a-74cd-11ef-bceb-c7b262605968","modified":"2024-09-17T08:19:38.042575Z","created":"2024-09-17T08:17:53.722837Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-17T08:19:43.822 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 3 2024-09-17T08:19:44.393 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:44 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:44.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:44 smithi193 ceph-mon[30002]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:44.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:44 smithi193 ceph-mon[30002]: from='client.? 172.21.15.203:0/1489402652' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:19:44.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:44 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:19:44.394 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:44 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:44.394 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:44 smithi203 ceph-mon[30259]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:44.394 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:44 smithi203 ceph-mon[30259]: from='client.? 172.21.15.203:0/1489402652' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:19:44.394 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:44 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:19:44.416 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-17T08:19:44.417 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-17T08:19:44.440 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:44 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:44.440 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:44 smithi047 ceph-mon[25866]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:44.441 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:44 smithi047 ceph-mon[25866]: from='client.? 172.21.15.203:0/1489402652' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:19:44.441 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:44 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:19:44.632 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:45.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:45 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:45.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:45 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:45.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:45 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:45.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:45 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:45.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:45 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:45.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:45 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:19:45.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:45 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:45.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:45 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:45.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:45 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:45.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:45 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:45.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:45 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:45.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:45 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:19:45.607 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:45 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:45.607 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:45 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:45.607 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:45 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:45.607 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:45 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:45.607 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:45 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:45.607 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:45 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:19:46.421 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-17T08:19:46.421 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-17T08:19:46.421 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-17T08:19:46.421 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: Updating smithi047:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:19:46.421 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: Updating smithi193:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:19:46.421 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: Updating smithi203:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:19:46.421 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.421 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.421 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.421 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: Reconfiguring mon.a (unknown last config time)... 2024-09-17T08:19:46.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:19:46.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:19:46.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:46.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: Reconfiguring daemon mon.a on smithi047 2024-09-17T08:19:46.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:46 smithi047 ceph-mon[25866]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:46.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-17T08:19:46.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-17T08:19:46.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-17T08:19:46.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: Updating smithi047:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:19:46.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: Updating smithi193:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:19:46.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: Updating smithi203:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:19:46.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: Reconfiguring mon.a (unknown last config time)... 2024-09-17T08:19:46.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:19:46.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:19:46.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:46.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: Reconfiguring daemon mon.a on smithi047 2024-09-17T08:19:46.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:46 smithi203 ceph-mon[30259]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:46.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-17T08:19:46.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-17T08:19:46.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-17T08:19:46.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: Updating smithi047:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:19:46.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: Updating smithi193:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:19:46.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: Updating smithi203:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:19:46.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.645 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.645 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.645 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.645 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.645 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.645 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.645 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.645 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.645 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:46.645 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: Reconfiguring mon.a (unknown last config time)... 2024-09-17T08:19:46.645 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:19:46.645 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:19:46.645 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:46.645 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: Reconfiguring daemon mon.a on smithi047 2024-09-17T08:19:46.646 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:46 smithi193 ceph-mon[30002]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:46.795 INFO:teuthology.orchestra.run.smithi047.stdout:# minimal ceph.conf for 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:19:46.796 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2024-09-17T08:19:46.796 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:19:46.796 INFO:teuthology.orchestra.run.smithi047.stdout: mon_host = [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] 2024-09-17T08:19:47.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:47 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/3282137074' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:47.570 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-17T08:19:47.570 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-17T08:19:47.570 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.conf 2024-09-17T08:19:47.642 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-17T08:19:47.642 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:19:47.711 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-17T08:19:47.711 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.conf 2024-09-17T08:19:47.738 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:47 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/3282137074' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:47.744 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-17T08:19:47.744 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:19:47.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:47 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/3282137074' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:47.812 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-17T08:19:47.812 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.conf 2024-09-17T08:19:47.845 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-17T08:19:47.845 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:19:47.914 INFO:tasks.cephadm:Adding mgr.y on smithi047 2024-09-17T08:19:47.915 INFO:tasks.cephadm:Adding mgr.x on smithi193 2024-09-17T08:19:47.915 INFO:tasks.cephadm:Adding mgr.z on smithi203 2024-09-17T08:19:47.915 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi047=y;smithi193=x;smithi203=z' 2024-09-17T08:19:48.145 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T08:19:48.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:48 smithi203 ceph-mon[30259]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:48.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:48 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:48.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:48 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:48.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:48 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:48.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:48 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:19:48.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:48 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:19:48.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:48 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:48.926 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:48 smithi193 ceph-mon[30002]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:48.927 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:48 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:48.927 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:48 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:48.927 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:48 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:48.927 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:48 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:19:48.927 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:48 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:19:48.927 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:48 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:49.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:48 smithi047 ceph-mon[25866]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:49.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:48 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:49.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:48 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:49.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:48 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:49.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:48 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:19:49.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:48 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:19:49.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:48 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:49.773 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:49 smithi203 ceph-mon[30259]: Reconfiguring mon.b (monmap changed)... 2024-09-17T08:19:49.773 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:49 smithi203 ceph-mon[30259]: Reconfiguring daemon mon.b on smithi193 2024-09-17T08:19:49.773 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:49 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:49.773 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:49 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:49.773 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:49 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:19:49.773 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:49 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:19:49.773 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:49 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:49.839 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mgr update... 2024-09-17T08:19:50.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:49 smithi047 ceph-mon[25866]: Reconfiguring mon.b (monmap changed)... 2024-09-17T08:19:50.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:49 smithi047 ceph-mon[25866]: Reconfiguring daemon mon.b on smithi193 2024-09-17T08:19:50.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:49 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:50.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:49 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:50.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:49 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:19:50.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:49 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:19:50.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:49 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:50.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:49 smithi193 ceph-mon[30002]: Reconfiguring mon.b (monmap changed)... 2024-09-17T08:19:50.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:49 smithi193 ceph-mon[30002]: Reconfiguring daemon mon.b on smithi193 2024-09-17T08:19:50.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:49 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:50.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:49 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:50.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:49 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:19:50.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:49 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:19:50.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:49 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:50.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:50 smithi203 ceph-mon[30259]: Reconfiguring mon.c (monmap changed)... 2024-09-17T08:19:50.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:50 smithi203 ceph-mon[30259]: Reconfiguring daemon mon.c on smithi203 2024-09-17T08:19:50.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:50 smithi203 ceph-mon[30259]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:50.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:50 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:51.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:50 smithi047 ceph-mon[25866]: Reconfiguring mon.c (monmap changed)... 2024-09-17T08:19:51.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:50 smithi047 ceph-mon[25866]: Reconfiguring daemon mon.c on smithi203 2024-09-17T08:19:51.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:50 smithi047 ceph-mon[25866]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:51.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:50 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:51.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:50 smithi193 ceph-mon[30002]: Reconfiguring mon.c (monmap changed)... 2024-09-17T08:19:51.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:50 smithi193 ceph-mon[30002]: Reconfiguring daemon mon.c on smithi203 2024-09-17T08:19:51.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:50 smithi193 ceph-mon[30002]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:51.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:50 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:51.226 DEBUG:teuthology.orchestra.run.smithi193:mgr.x> sudo journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mgr.x.service 2024-09-17T08:19:51.229 DEBUG:teuthology.orchestra.run.smithi203:mgr.z> sudo journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mgr.z.service 2024-09-17T08:19:51.232 INFO:tasks.cephadm:Deploying OSDs... 2024-09-17T08:19:51.232 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-17T08:19:51.232 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T08:19:51.248 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-17T08:19:51.249 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/nvme4n1 2024-09-17T08:19:51.304 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/nvme4n1 2024-09-17T08:19:51.304 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T08:19:51.304 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 1040 Links: 1 Device type: 103,8 2024-09-17T08:19:51.304 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T08:19:51.304 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T08:19:51.304 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-17 08:19:00.286781741 +0000 2024-09-17T08:19:51.305 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-17 08:17:03.336365279 +0000 2024-09-17T08:19:51.305 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-17 08:17:03.336365279 +0000 2024-09-17T08:19:51.305 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-17 08:17:03.333365346 +0000 2024-09-17T08:19:51.305 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-17T08:19:51.369 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-17T08:19:51.369 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-17T08:19:51.369 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000114955 s, 4.5 MB/s 2024-09-17T08:19:51.370 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-17T08:19:51.427 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/nvme3n1 2024-09-17T08:19:51.483 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/nvme3n1 2024-09-17T08:19:51.483 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T08:19:51.483 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 1027 Links: 1 Device type: 103,6 2024-09-17T08:19:51.484 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T08:19:51.484 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T08:19:51.484 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-17 08:19:00.277781938 +0000 2024-09-17T08:19:51.484 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-17 08:17:03.165369084 +0000 2024-09-17T08:19:51.484 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-17 08:17:03.165369084 +0000 2024-09-17T08:19:51.484 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-17 08:17:03.162369151 +0000 2024-09-17T08:19:51.484 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-17T08:19:51.547 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-17T08:19:51.547 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-17T08:19:51.548 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000132234 s, 3.9 MB/s 2024-09-17T08:19:51.549 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-17T08:19:51.606 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/nvme2n1 2024-09-17T08:19:51.663 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/nvme2n1 2024-09-17T08:19:51.663 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T08:19:51.663 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 1015 Links: 1 Device type: 103,4 2024-09-17T08:19:51.663 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T08:19:51.663 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T08:19:51.663 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-17 08:19:00.268782135 +0000 2024-09-17T08:19:51.663 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-17 08:17:03.025372200 +0000 2024-09-17T08:19:51.663 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-17 08:17:03.025372200 +0000 2024-09-17T08:19:51.663 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-17 08:17:03.022372267 +0000 2024-09-17T08:19:51.664 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-17T08:19:51.728 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-17T08:19:51.728 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-17T08:19:51.728 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.00013038 s, 3.9 MB/s 2024-09-17T08:19:51.729 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-17T08:19:51.787 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/nvme1n1 2024-09-17T08:19:51.843 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/nvme1n1 2024-09-17T08:19:51.843 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T08:19:51.843 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 1004 Links: 1 Device type: 103,2 2024-09-17T08:19:51.843 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T08:19:51.843 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T08:19:51.844 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-17 08:19:00.260782310 +0000 2024-09-17T08:19:51.844 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-17 08:17:02.884375338 +0000 2024-09-17T08:19:51.844 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-17 08:17:02.884375338 +0000 2024-09-17T08:19:51.844 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-17 08:17:02.880375427 +0000 2024-09-17T08:19:51.844 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-17T08:19:51.907 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-17T08:19:51.907 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-17T08:19:51.907 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000115835 s, 4.4 MB/s 2024-09-17T08:19:51.908 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-17T08:19:51.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:51 smithi203 ceph-mon[30259]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=y;smithi193=x;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:19:51.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:51 smithi203 ceph-mon[30259]: Saving service mgr spec with placement smithi047=y;smithi193=x;smithi203=z;count:3 2024-09-17T08:19:51.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:51 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:51.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:51 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:51.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:51 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:51.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:51 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:51.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:51 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:19:51.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:51 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:51.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:51 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:19:51.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:51 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T08:19:51.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:51 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:19:51.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:51 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:51.965 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-17T08:19:51.965 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T08:19:51.986 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-17T08:19:51.986 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/nvme4n1 2024-09-17T08:19:52.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:51 smithi047 ceph-mon[25866]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=y;smithi193=x;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:19:52.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:51 smithi047 ceph-mon[25866]: Saving service mgr spec with placement smithi047=y;smithi193=x;smithi203=z;count:3 2024-09-17T08:19:52.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:51 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:52.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:51 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:52.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:51 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:52.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:51 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:52.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:51 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:19:52.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:51 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:52.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:51 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:19:52.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:51 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T08:19:52.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:51 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:19:52.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:51 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:52.044 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/nvme4n1 2024-09-17T08:19:52.044 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T08:19:52.044 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 1040 Links: 1 Device type: 103,8 2024-09-17T08:19:52.044 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T08:19:52.044 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T08:19:52.045 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-17 08:19:12.250578205 +0000 2024-09-17T08:19:52.045 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-17 08:17:04.973560324 +0000 2024-09-17T08:19:52.045 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-17 08:17:04.973560324 +0000 2024-09-17T08:19:52.045 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-17 08:17:04.968560401 +0000 2024-09-17T08:19:52.045 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-17T08:19:52.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:51 smithi193 ceph-mon[30002]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=y;smithi193=x;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:19:52.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:51 smithi193 ceph-mon[30002]: Saving service mgr spec with placement smithi047=y;smithi193=x;smithi203=z;count:3 2024-09-17T08:19:52.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:51 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:52.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:51 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:52.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:51 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:52.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:51 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:52.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:51 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:19:52.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:51 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:52.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:51 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:19:52.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:51 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T08:19:52.110 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:51 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:19:52.110 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:51 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:52.111 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-17T08:19:52.111 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-17T08:19:52.111 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000247165 s, 2.1 MB/s 2024-09-17T08:19:52.113 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-17T08:19:52.172 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/nvme3n1 2024-09-17T08:19:52.229 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/nvme3n1 2024-09-17T08:19:52.230 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T08:19:52.230 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 1027 Links: 1 Device type: 103,6 2024-09-17T08:19:52.230 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T08:19:52.230 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T08:19:52.230 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-17 08:19:12.240578357 +0000 2024-09-17T08:19:52.230 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-17 08:17:04.834562489 +0000 2024-09-17T08:19:52.230 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-17 08:17:04.834562489 +0000 2024-09-17T08:19:52.230 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-17 08:17:04.830562552 +0000 2024-09-17T08:19:52.230 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-17T08:19:52.295 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-17T08:19:52.295 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-17T08:19:52.295 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.00027215 s, 1.9 MB/s 2024-09-17T08:19:52.297 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-17T08:19:52.354 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/nvme2n1 2024-09-17T08:19:52.411 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/nvme2n1 2024-09-17T08:19:52.411 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T08:19:52.411 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 1015 Links: 1 Device type: 103,4 2024-09-17T08:19:52.411 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T08:19:52.412 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T08:19:52.412 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-17 08:19:12.230578510 +0000 2024-09-17T08:19:52.412 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-17 08:17:04.695564655 +0000 2024-09-17T08:19:52.412 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-17 08:17:04.695564655 +0000 2024-09-17T08:19:52.412 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-17 08:17:04.690564733 +0000 2024-09-17T08:19:52.413 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-17T08:19:52.477 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-17T08:19:52.477 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-17T08:19:52.477 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.00124159 s, 412 kB/s 2024-09-17T08:19:52.478 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-17T08:19:52.534 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/nvme1n1 2024-09-17T08:19:52.589 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/nvme1n1 2024-09-17T08:19:52.589 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T08:19:52.589 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 1004 Links: 1 Device type: 103,2 2024-09-17T08:19:52.589 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T08:19:52.589 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T08:19:52.590 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-17 08:19:12.219578677 +0000 2024-09-17T08:19:52.590 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-17 08:17:04.541567055 +0000 2024-09-17T08:19:52.590 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-17 08:17:04.541567055 +0000 2024-09-17T08:19:52.590 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-17 08:17:04.536567133 +0000 2024-09-17T08:19:52.590 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-17T08:19:52.652 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-17T08:19:52.653 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-17T08:19:52.653 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000235145 s, 2.2 MB/s 2024-09-17T08:19:52.654 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-17T08:19:52.710 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-17T08:19:52.710 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T08:19:52.729 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-17T08:19:52.729 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/nvme4n1 2024-09-17T08:19:52.785 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/nvme4n1 2024-09-17T08:19:52.785 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T08:19:52.785 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 1039 Links: 1 Device type: 103,8 2024-09-17T08:19:52.785 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T08:19:52.785 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T08:19:52.785 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-09-17 08:19:20.351810718 +0000 2024-09-17T08:19:52.785 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-09-17 08:17:06.589211894 +0000 2024-09-17T08:19:52.785 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-09-17 08:17:06.589211894 +0000 2024-09-17T08:19:52.785 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-09-17 08:17:06.585211996 +0000 2024-09-17T08:19:52.786 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-17T08:19:52.847 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:52 smithi203 ceph-mon[30259]: Deploying daemon mgr.z on smithi203 2024-09-17T08:19:52.847 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:52 smithi203 ceph-mon[30259]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:52.853 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-09-17T08:19:52.853 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-09-17T08:19:52.853 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000228704 s, 2.2 MB/s 2024-09-17T08:19:52.855 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-17T08:19:52.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:52 smithi193 ceph-mon[30002]: Deploying daemon mgr.z on smithi203 2024-09-17T08:19:52.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:52 smithi193 ceph-mon[30002]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:52.917 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/nvme3n1 2024-09-17T08:19:52.972 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/nvme3n1 2024-09-17T08:19:52.972 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T08:19:52.972 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 1027 Links: 1 Device type: 103,6 2024-09-17T08:19:52.972 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T08:19:52.972 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T08:19:52.972 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-09-17 08:19:20.341810967 +0000 2024-09-17T08:19:52.972 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-09-17 08:17:06.449215469 +0000 2024-09-17T08:19:52.972 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-09-17 08:17:06.449215469 +0000 2024-09-17T08:19:52.972 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-09-17 08:17:06.445215571 +0000 2024-09-17T08:19:52.973 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-17T08:19:53.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:52 smithi047 ceph-mon[25866]: Deploying daemon mgr.z on smithi203 2024-09-17T08:19:53.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:52 smithi047 ceph-mon[25866]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:53.044 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-09-17T08:19:53.044 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-09-17T08:19:53.044 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000224459 s, 2.3 MB/s 2024-09-17T08:19:53.046 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-17T08:19:53.103 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/nvme2n1 2024-09-17T08:19:53.159 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/nvme2n1 2024-09-17T08:19:53.159 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T08:19:53.159 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 1015 Links: 1 Device type: 103,4 2024-09-17T08:19:53.159 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T08:19:53.159 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T08:19:53.160 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-09-17 08:19:20.330811241 +0000 2024-09-17T08:19:53.160 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-09-17 08:17:06.310219017 +0000 2024-09-17T08:19:53.160 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-09-17 08:17:06.310219017 +0000 2024-09-17T08:19:53.160 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-09-17 08:17:06.304219170 +0000 2024-09-17T08:19:53.160 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-17T08:19:53.224 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-09-17T08:19:53.224 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-09-17T08:19:53.224 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000215024 s, 2.4 MB/s 2024-09-17T08:19:53.225 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-17T08:19:53.281 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/nvme1n1 2024-09-17T08:19:53.336 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/nvme1n1 2024-09-17T08:19:53.337 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T08:19:53.337 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 1004 Links: 1 Device type: 103,2 2024-09-17T08:19:53.337 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T08:19:53.337 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T08:19:53.337 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-09-17 08:19:20.320811490 +0000 2024-09-17T08:19:53.337 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-09-17 08:17:06.163222770 +0000 2024-09-17T08:19:53.337 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-09-17 08:17:06.163222770 +0000 2024-09-17T08:19:53.337 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-09-17 08:17:06.158222898 +0000 2024-09-17T08:19:53.337 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-17T08:19:53.400 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-09-17T08:19:53.400 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-09-17T08:19:53.400 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000212817 s, 2.4 MB/s 2024-09-17T08:19:53.402 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-17T08:19:53.458 INFO:tasks.cephadm:Deploying osd.0 on smithi047 with /dev/nvme1n1... 2024-09-17T08:19:53.458 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme1n1 2024-09-17T08:19:53.667 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:53.876 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:53 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:53.727+0000 7fbceaeaa040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:19:53.876 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:53 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:53.874+0000 7fbceaeaa040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:19:54.222 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:53 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:53.940+0000 7fbceaeaa040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:19:54.222 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:54 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:54.083+0000 7fbceaeaa040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:19:54.222 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:54 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:54.221+0000 7fbceaeaa040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:19:54.784 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:54 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:54.529+0000 7fbceaeaa040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:19:54.784 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:54 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:54.590+0000 7fbceaeaa040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:19:54.784 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:54 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:54.771+0000 7fbceaeaa040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:19:54.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:54 smithi203 ceph-mon[30259]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:54.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:54 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:54.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:54 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:54.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:54 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:54.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:54 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:19:54.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:54 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T08:19:54.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:54 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:19:54.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:54 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:54.916 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:54 smithi193 ceph-mon[30002]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:54.916 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:54 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:54.916 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:54 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:54.917 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:54 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:54.917 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:54 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:19:54.917 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:54 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T08:19:54.917 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:54 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:19:54.917 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:54 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:54.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:54 smithi047 ceph-mon[25866]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:54.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:54 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:54.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:54 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:54.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:54 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:54.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:54 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:19:54.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:54 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T08:19:54.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:54 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:19:54.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:54 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:55.561 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:55 smithi193 systemd[1]: Starting Ceph mgr.x for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:19:55.688 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/nvme1n1 2024-09-17T08:19:55.688 INFO:teuthology.orchestra.run.smithi047.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-17T08:19:55.691 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2024-09-17T08:19:55.702 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-09-17T08:19:55.702 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-09-17T08:19:55.703 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0093762 s, 1.1 GB/s 2024-09-17T08:19:55.703 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-09-17T08:19:55.816 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:55 smithi193 ceph-mon[30002]: Deploying daemon mgr.x on smithi193 2024-09-17T08:19:55.816 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:55 smithi193 podman[31420]: 2024-09-17 08:19:55.5605187 +0000 UTC m=+0.103304248 container create d54a413ea2d5429f20bfe770fe1bdf29b72f49e2d23f3b8e4df24806e6e4721b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:19:55.816 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:55 smithi193 podman[31420]: 2024-09-17 08:19:55.473044743 +0000 UTC m=+0.015830248 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:19:55.816 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:55 smithi193 podman[31420]: 2024-09-17 08:19:55.787840643 +0000 UTC m=+0.330626151 container init d54a413ea2d5429f20bfe770fe1bdf29b72f49e2d23f3b8e4df24806e6e4721b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-17T08:19:55.817 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:55 smithi193 podman[31420]: 2024-09-17 08:19:55.792529937 +0000 UTC m=+0.335315443 container start d54a413ea2d5429f20bfe770fe1bdf29b72f49e2d23f3b8e4df24806e6e4721b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-17T08:19:55.817 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:55 smithi193 ceph-mgr[31458]: -- 172.21.15.193:0/722760353 <== mon.2 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b011782d00 con 0x55b011756800 2024-09-17T08:19:56.007 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:55 smithi047 ceph-mon[25866]: Deploying daemon mgr.x on smithi193 2024-09-17T08:19:56.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:55 smithi203 ceph-mon[30259]: Deploying daemon mgr.x on smithi193 2024-09-17T08:19:56.034 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:55 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:55.636+0000 7fbceaeaa040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:19:56.034 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:55 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:55.703+0000 7fbceaeaa040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:19:56.131 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:55 smithi193 bash[31420]: d54a413ea2d5429f20bfe770fe1bdf29b72f49e2d23f3b8e4df24806e6e4721b 2024-09-17T08:19:56.131 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:55 smithi193 systemd[1]: Started Ceph mgr.x for 3a66518a-74cd-11ef-bceb-c7b262605968. 2024-09-17T08:19:56.131 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:55 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:55.917+0000 7f2c58440040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:19:56.131 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:55 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:55.974+0000 7f2c58440040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:19:56.131 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:56 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:56.129+0000 7f2c58440040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:19:56.287 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi047:/dev/nvme1n1 2024-09-17T08:19:56.393 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:56 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:56.198+0000 7f2c58440040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:19:56.394 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:56 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:56.362+0000 7f2c58440040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:19:56.484 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:56.728 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:56 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:56.451+0000 7fbceaeaa040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:19:56.728 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:56 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:56.567+0000 7fbceaeaa040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:19:56.728 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:56 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:56.726+0000 7fbceaeaa040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:19:56.728 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:56 smithi203 ceph-mon[30259]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:56.729 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:56 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:56.729 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:56 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:56.729 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:56 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:56.729 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:56 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:56.729 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:56 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:56.819 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:56 smithi193 ceph-mon[30002]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:56.819 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:56 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:56.819 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:56 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:56.820 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:56 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:56.820 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:56 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:56.820 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:56 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:56.820 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:56 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:56.502+0000 7f2c58440040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:19:56.820 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:56 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:56.817+0000 7f2c58440040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:19:56.910 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:56 smithi047 ceph-mon[25866]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:56.910 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:56 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:56.911 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:56 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:56.911 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:56 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:56.911 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:56 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:56.911 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:56 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:57.006 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:56 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:56.788+0000 7fbceaeaa040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:19:57.006 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:56 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:56.937+0000 7fbceaeaa040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:19:57.006 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:57 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:57.005+0000 7fbceaeaa040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:19:57.143 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:56 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:56.878+0000 7f2c58440040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:19:57.143 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:57 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:57.067+0000 7f2c58440040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:19:57.284 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:57 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:57.081+0000 7fbceaeaa040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:19:57.284 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:57 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:57.248+0000 7fbceaeaa040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:19:57.598 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:57 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:57.315+0000 7fbceaeaa040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:19:57.598 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:57 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:57.384+0000 7fbceaeaa040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:19:57.598 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:57 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:57.472+0000 7fbceaeaa040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:19:57.598 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:57 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:57.535+0000 7fbceaeaa040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:19:57.598 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:57 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:57.596+0000 7fbceaeaa040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:19:58.244 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:58 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:58.244 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:58 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:58.244 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:58 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:58.244 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:58 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:58.244 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:58 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:58.245 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:58 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:19:58.245 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:58 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:58.245 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:58 smithi047 ceph-mon[25866]: Reconfiguring mgr.y (unknown last config time)... 2024-09-17T08:19:58.245 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:58 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:19:58.245 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:58 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:19:58.245 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:58 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:58.245 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:58 smithi047 ceph-mon[25866]: Reconfiguring daemon mgr.y on smithi047 2024-09-17T08:19:58.245 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:58 smithi047 ceph-mon[25866]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:58.245 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:58 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:58.394 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:57 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:57.924+0000 7f2c58440040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:19:58.394 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:57 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:57.988+0000 7f2c58440040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:19:58.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:58 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:58.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:58 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:58.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:58 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:58.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:58 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:58.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:58 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:58.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:58 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:19:58.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:58 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:58.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:58 smithi193 ceph-mon[30002]: Reconfiguring mgr.y (unknown last config time)... 2024-09-17T08:19:58.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:58 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:19:58.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:58 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:19:58.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:58 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:58.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:58 smithi193 ceph-mon[30002]: Reconfiguring daemon mgr.y on smithi047 2024-09-17T08:19:58.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:58 smithi193 ceph-mon[30002]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:58.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:58 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:58.402 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:58 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:58.402 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:58 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:58.402 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:58 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:58.402 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:58 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:58.402 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:58 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:58.402 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:58 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:19:58.402 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:58 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:58.402 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:58 smithi203 ceph-mon[30259]: Reconfiguring mgr.y (unknown last config time)... 2024-09-17T08:19:58.402 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:58 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:19:58.403 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:58 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:19:58.403 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:58 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:58.403 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:58 smithi203 ceph-mon[30259]: Reconfiguring daemon mgr.y on smithi047 2024-09-17T08:19:58.403 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:58 smithi203 ceph-mon[30259]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:19:58.403 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:58 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:19:58.784 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:19:58 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:19:58.401+0000 7fbceaeaa040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:19:59.000 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:58 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:58.692+0000 7f2c58440040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:19:59.001 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:58 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:58.803+0000 7f2c58440040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:19:59.001 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:58 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:58.941+0000 7f2c58440040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:19:59.001 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:59 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:59.000+0000 7f2c58440040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:19:59.202 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:59 smithi047 ceph-mon[25866]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:19:59.218 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:59 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:19:59.218 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:59 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:19:59.218 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:59 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:59.218 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:59 smithi047 ceph-mon[25866]: Standby manager daemon z started 2024-09-17T08:19:59.218 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:59 smithi047 ceph-mon[25866]: from='mgr.? 172.21.15.203:0/2425766643' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-17T08:19:59.218 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:59 smithi047 ceph-mon[25866]: from='mgr.? 172.21.15.203:0/2425766643' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:19:59.218 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:59 smithi047 ceph-mon[25866]: from='mgr.? 172.21.15.203:0/2425766643' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-17T08:19:59.218 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:19:59 smithi047 ceph-mon[25866]: from='mgr.? 172.21.15.203:0/2425766643' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:19:59.270 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:59 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:59.145+0000 7f2c58440040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:19:59.270 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:59 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:59.211+0000 7f2c58440040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:19:59.270 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:59 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:59.269+0000 7f2c58440040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:19:59.270 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:59 smithi193 ceph-mon[30002]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:19:59.270 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:59 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:19:59.270 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:59 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:19:59.270 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:59 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:59.270 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:59 smithi193 ceph-mon[30002]: Standby manager daemon z started 2024-09-17T08:19:59.271 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:59 smithi193 ceph-mon[30002]: from='mgr.? 172.21.15.203:0/2425766643' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-17T08:19:59.271 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:59 smithi193 ceph-mon[30002]: from='mgr.? 172.21.15.203:0/2425766643' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:19:59.271 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:59 smithi193 ceph-mon[30002]: from='mgr.? 172.21.15.203:0/2425766643' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-17T08:19:59.271 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:19:59 smithi193 ceph-mon[30002]: from='mgr.? 172.21.15.203:0/2425766643' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:19:59.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:59 smithi203 ceph-mon[30259]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:19:59.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:59 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:19:59.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:59 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:19:59.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:59 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:59.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:59 smithi203 ceph-mon[30259]: Standby manager daemon z started 2024-09-17T08:19:59.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:59 smithi203 ceph-mon[30259]: from='mgr.? 172.21.15.203:0/2425766643' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-17T08:19:59.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:59 smithi203 ceph-mon[30259]: from='mgr.? 172.21.15.203:0/2425766643' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:19:59.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:59 smithi203 ceph-mon[30259]: from='mgr.? 172.21.15.203:0/2425766643' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-17T08:19:59.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:19:59 smithi203 ceph-mon[30259]: from='mgr.? 172.21.15.203:0/2425766643' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:19:59.555 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:59 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:59.427+0000 7f2c58440040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:19:59.555 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:59 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:59.490+0000 7f2c58440040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:19:59.555 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:59 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:59.554+0000 7f2c58440040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:19:59.893 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:59 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:59.634+0000 7f2c58440040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:19:59.894 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:59 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:59.691+0000 7f2c58440040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:19:59.894 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:19:59 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:19:59.749+0000 7f2c58440040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:20:00.393 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:00 smithi193 ceph-mon[30002]: mgrmap e14: y(active, since 71s), standbys: z 2024-09-17T08:20:00.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:00 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-17T08:20:00.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:00 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:00.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:00 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:00.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:00 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:20:00.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:00 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:00.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:00 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:20:00.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:00 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:00.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:00 smithi193 ceph-mon[30002]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:00.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:00 smithi193 ceph-mon[30002]: overall HEALTH_OK 2024-09-17T08:20:00.430 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:00 smithi047 ceph-mon[25866]: mgrmap e14: y(active, since 71s), standbys: z 2024-09-17T08:20:00.430 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:00 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-17T08:20:00.430 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:00 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:00.430 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:00 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:00.430 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:00 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:20:00.430 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:00 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:00.430 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:00 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:20:00.430 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:00 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:00.430 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:00 smithi047 ceph-mon[25866]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:00.430 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:00 smithi047 ceph-mon[25866]: overall HEALTH_OK 2024-09-17T08:20:00.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:00 smithi203 ceph-mon[30259]: mgrmap e14: y(active, since 71s), standbys: z 2024-09-17T08:20:00.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:00 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-17T08:20:00.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:00 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:00.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:00 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:00.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:00 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:20:00.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:00 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:00.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:00 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:20:00.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:00 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:00.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:00 smithi203 ceph-mon[30259]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:00.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:00 smithi203 ceph-mon[30259]: overall HEALTH_OK 2024-09-17T08:20:00.894 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:20:00 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:20:00.510+0000 7f2c58440040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:20:01.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:01 smithi047 ceph-mon[25866]: Standby manager daemon x started 2024-09-17T08:20:01.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:01 smithi047 ceph-mon[25866]: from='mgr.? 172.21.15.193:0/3249829447' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T08:20:01.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:01 smithi047 ceph-mon[25866]: from='mgr.? 172.21.15.193:0/3249829447' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:20:01.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:01 smithi047 ceph-mon[25866]: from='mgr.? 172.21.15.193:0/3249829447' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T08:20:01.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:01 smithi047 ceph-mon[25866]: from='mgr.? 172.21.15.193:0/3249829447' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:20:01.643 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:01 smithi193 ceph-mon[30002]: Standby manager daemon x started 2024-09-17T08:20:01.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:01 smithi193 ceph-mon[30002]: from='mgr.? 172.21.15.193:0/3249829447' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T08:20:01.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:01 smithi193 ceph-mon[30002]: from='mgr.? 172.21.15.193:0/3249829447' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:20:01.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:01 smithi193 ceph-mon[30002]: from='mgr.? 172.21.15.193:0/3249829447' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T08:20:01.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:01 smithi193 ceph-mon[30002]: from='mgr.? 172.21.15.193:0/3249829447' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:20:01.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:01 smithi203 ceph-mon[30259]: Standby manager daemon x started 2024-09-17T08:20:01.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:01 smithi203 ceph-mon[30259]: from='mgr.? 172.21.15.193:0/3249829447' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T08:20:01.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:01 smithi203 ceph-mon[30259]: from='mgr.? 172.21.15.193:0/3249829447' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:20:01.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:01 smithi203 ceph-mon[30259]: from='mgr.? 172.21.15.193:0/3249829447' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T08:20:01.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:01 smithi203 ceph-mon[30259]: from='mgr.? 172.21.15.193:0/3249829447' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:20:02.643 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:02 smithi193 ceph-mon[30002]: mgrmap e15: y(active, since 73s), standbys: z, x 2024-09-17T08:20:02.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:02 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T08:20:02.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:02 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/1073906617' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ec4352f-9153-4631-9033-a6203fe48eed"}]: dispatch 2024-09-17T08:20:02.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:02 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/1073906617' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ec4352f-9153-4631-9033-a6203fe48eed"}]': finished 2024-09-17T08:20:02.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:02 smithi193 ceph-mon[30002]: osdmap e5: 1 total, 0 up, 1 in 2024-09-17T08:20:02.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:02 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:20:02.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:02 smithi193 ceph-mon[30002]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:02.651 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:02 smithi047 ceph-mon[25866]: mgrmap e15: y(active, since 73s), standbys: z, x 2024-09-17T08:20:02.652 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:02 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T08:20:02.652 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:02 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/1073906617' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ec4352f-9153-4631-9033-a6203fe48eed"}]: dispatch 2024-09-17T08:20:02.652 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:02 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/1073906617' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ec4352f-9153-4631-9033-a6203fe48eed"}]': finished 2024-09-17T08:20:02.652 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:02 smithi047 ceph-mon[25866]: osdmap e5: 1 total, 0 up, 1 in 2024-09-17T08:20:02.652 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:02 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:20:02.652 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:02 smithi047 ceph-mon[25866]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:02.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:02 smithi203 ceph-mon[30259]: mgrmap e15: y(active, since 73s), standbys: z, x 2024-09-17T08:20:02.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:02 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T08:20:02.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:02 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/1073906617' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ec4352f-9153-4631-9033-a6203fe48eed"}]: dispatch 2024-09-17T08:20:02.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:02 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/1073906617' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ec4352f-9153-4631-9033-a6203fe48eed"}]': finished 2024-09-17T08:20:02.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:02 smithi203 ceph-mon[30259]: osdmap e5: 1 total, 0 up, 1 in 2024-09-17T08:20:02.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:02 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:20:02.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:02 smithi203 ceph-mon[30259]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:03.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:03 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/2094375925' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:20:03.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:03 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/2094375925' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:20:03.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:03 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/2094375925' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:20:04.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:04 smithi203 ceph-mon[30259]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:04.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:04 smithi047 ceph-mon[25866]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:04.794 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:04 smithi193 ceph-mon[30002]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:05.986 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:05 smithi193 ceph-mon[30002]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:06.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:05 smithi203 ceph-mon[30259]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:06.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:05 smithi047 ceph-mon[25866]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:08.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:07 smithi203 ceph-mon[30259]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:08.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:07 smithi047 ceph-mon[25866]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:08.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:07 smithi193 ceph-mon[30002]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:08.991 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:08 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-17T08:20:08.991 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:08 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:08.991 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:08 smithi047 ceph-mon[25866]: Deploying daemon osd.0 on smithi047 2024-09-17T08:20:09.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:08 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-17T08:20:09.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:08 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:09.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:08 smithi203 ceph-mon[30259]: Deploying daemon osd.0 on smithi047 2024-09-17T08:20:09.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:08 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-17T08:20:09.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:08 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:09.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:08 smithi193 ceph-mon[30002]: Deploying daemon osd.0 on smithi047 2024-09-17T08:20:09.875 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:09 smithi047 ceph-mon[25866]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:10.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:09 smithi203 ceph-mon[30259]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:10.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:09 smithi193 ceph-mon[30002]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:12.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:11 smithi203 ceph-mon[30259]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:12.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:11 smithi047 ceph-mon[25866]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:12.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:11 smithi193 ceph-mon[30002]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:13.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:13 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:20:13.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:13 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:13.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:13 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:13.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:13 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:20:13.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:13 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:13.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:13 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:13.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:13 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:20:13.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:13 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:13.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:13 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:14.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:14 smithi047 ceph-mon[25866]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:14.393 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:14 smithi193 ceph-mon[30002]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:14.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:14 smithi203 ceph-mon[30259]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:16.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:16 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:16.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:16 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:16.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:16 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:16.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:16 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:20:16.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:16 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:16.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:16 smithi047 ceph-mon[25866]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:16.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:16 smithi047 ceph-mon[25866]: from='osd.0 [v2:172.21.15.47:6802/619587640,v1:172.21.15.47:6803/619587640]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T08:20:16.551 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 0 on host 'smithi047' 2024-09-17T08:20:16.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:16 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:16.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:16 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:16.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:16 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:16.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:16 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:20:16.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:16 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:16.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:16 smithi203 ceph-mon[30259]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:16.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:16 smithi203 ceph-mon[30259]: from='osd.0 [v2:172.21.15.47:6802/619587640,v1:172.21.15.47:6803/619587640]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T08:20:16.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:16 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:16.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:16 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:16.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:16 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:16.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:16 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:20:16.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:16 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:16.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:16 smithi193 ceph-mon[30002]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:16.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:16 smithi193 ceph-mon[30002]: from='osd.0 [v2:172.21.15.47:6802/619587640,v1:172.21.15.47:6803/619587640]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T08:20:17.166 DEBUG:teuthology.orchestra.run.smithi047:osd.0> sudo journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@osd.0.service 2024-09-17T08:20:17.169 INFO:tasks.cephadm:Deploying osd.1 on smithi193 with /dev/nvme1n1... 2024-09-17T08:20:17.169 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme1n1 2024-09-17T08:20:17.371 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:20:17.474 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:17 smithi193 ceph-mon[30002]: from='osd.0 [v2:172.21.15.47:6802/619587640,v1:172.21.15.47:6803/619587640]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-17T08:20:17.475 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:17 smithi193 ceph-mon[30002]: osdmap e6: 1 total, 0 up, 1 in 2024-09-17T08:20:17.475 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:17 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:20:17.475 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:17 smithi193 ceph-mon[30002]: from='osd.0 [v2:172.21.15.47:6802/619587640,v1:172.21.15.47:6803/619587640]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-17T08:20:17.475 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:17 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:20:17.475 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:17 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:17.475 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:17 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:17.745 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:17 smithi047 ceph-mon[25866]: from='osd.0 [v2:172.21.15.47:6802/619587640,v1:172.21.15.47:6803/619587640]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-17T08:20:17.745 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:17 smithi047 ceph-mon[25866]: osdmap e6: 1 total, 0 up, 1 in 2024-09-17T08:20:17.745 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:17 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:20:17.746 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:17 smithi047 ceph-mon[25866]: from='osd.0 [v2:172.21.15.47:6802/619587640,v1:172.21.15.47:6803/619587640]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-17T08:20:17.746 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:17 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:20:17.746 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:17 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:17.746 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:17 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:17.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:17 smithi203 ceph-mon[30259]: from='osd.0 [v2:172.21.15.47:6802/619587640,v1:172.21.15.47:6803/619587640]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-17T08:20:17.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:17 smithi203 ceph-mon[30259]: osdmap e6: 1 total, 0 up, 1 in 2024-09-17T08:20:17.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:17 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:20:17.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:17 smithi203 ceph-mon[30259]: from='osd.0 [v2:172.21.15.47:6802/619587640,v1:172.21.15.47:6803/619587640]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-17T08:20:17.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:17 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:20:17.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:17 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:17.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:17 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:18.016 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:20:17 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-0[44053]: 2024-09-17T08:20:17.744+0000 7faefcdb1640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T08:20:18.017 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:20:17 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-0[44053]: 2024-09-17T08:20:17.746+0000 7faef4bd2640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T08:20:18.588 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:18 smithi047 ceph-mon[25866]: from='osd.0 [v2:172.21.15.47:6802/619587640,v1:172.21.15.47:6803/619587640]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-17T08:20:18.588 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:18 smithi047 ceph-mon[25866]: osdmap e7: 1 total, 0 up, 1 in 2024-09-17T08:20:18.588 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:18 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:20:18.588 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:18 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:20:18.588 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:18 smithi047 ceph-mon[25866]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:18.588 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:18 smithi047 ceph-mon[25866]: from='osd.0 [v2:172.21.15.47:6802/619587640,v1:172.21.15.47:6803/619587640]' entity='osd.0' 2024-09-17T08:20:18.701 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:18 smithi193 ceph-mon[30002]: from='osd.0 [v2:172.21.15.47:6802/619587640,v1:172.21.15.47:6803/619587640]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-17T08:20:18.701 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:18 smithi193 ceph-mon[30002]: osdmap e7: 1 total, 0 up, 1 in 2024-09-17T08:20:18.701 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:18 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:20:18.701 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:18 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:20:18.701 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:18 smithi193 ceph-mon[30002]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:18.701 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:18 smithi193 ceph-mon[30002]: from='osd.0 [v2:172.21.15.47:6802/619587640,v1:172.21.15.47:6803/619587640]' entity='osd.0' 2024-09-17T08:20:18.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:18 smithi203 ceph-mon[30259]: from='osd.0 [v2:172.21.15.47:6802/619587640,v1:172.21.15.47:6803/619587640]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-17T08:20:18.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:18 smithi203 ceph-mon[30259]: osdmap e7: 1 total, 0 up, 1 in 2024-09-17T08:20:18.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:18 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:20:18.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:18 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:20:18.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:18 smithi203 ceph-mon[30259]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:20:18.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:18 smithi203 ceph-mon[30259]: from='osd.0 [v2:172.21.15.47:6802/619587640,v1:172.21.15.47:6803/619587640]' entity='osd.0' 2024-09-17T08:20:19.135 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/nvme1n1 2024-09-17T08:20:19.135 INFO:teuthology.orchestra.run.smithi193.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-17T08:20:19.139 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2024-09-17T08:20:19.154 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-09-17T08:20:19.154 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-09-17T08:20:19.154 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130965 s, 801 MB/s 2024-09-17T08:20:19.154 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-09-17T08:20:19.728 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:19 smithi193 ceph-mon[30002]: purged_snaps scrub starts 2024-09-17T08:20:19.728 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:19 smithi193 ceph-mon[30002]: purged_snaps scrub ok 2024-09-17T08:20:19.728 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:19 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:20:19.728 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:19 smithi193 ceph-mon[30002]: osd.0 [v2:172.21.15.47:6802/619587640,v1:172.21.15.47:6803/619587640] boot 2024-09-17T08:20:19.729 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:19 smithi193 ceph-mon[30002]: osdmap e8: 1 total, 1 up, 1 in 2024-09-17T08:20:19.729 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:19 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:20:19.751 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi193:/dev/nvme1n1 2024-09-17T08:20:19.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:19 smithi203 ceph-mon[30259]: purged_snaps scrub starts 2024-09-17T08:20:19.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:19 smithi203 ceph-mon[30259]: purged_snaps scrub ok 2024-09-17T08:20:19.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:19 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:20:19.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:19 smithi203 ceph-mon[30259]: osd.0 [v2:172.21.15.47:6802/619587640,v1:172.21.15.47:6803/619587640] boot 2024-09-17T08:20:19.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:19 smithi203 ceph-mon[30259]: osdmap e8: 1 total, 1 up, 1 in 2024-09-17T08:20:19.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:19 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:20:19.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:19 smithi047 ceph-mon[25866]: purged_snaps scrub starts 2024-09-17T08:20:19.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:19 smithi047 ceph-mon[25866]: purged_snaps scrub ok 2024-09-17T08:20:19.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:19 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:20:19.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:19 smithi047 ceph-mon[25866]: osd.0 [v2:172.21.15.47:6802/619587640,v1:172.21.15.47:6803/619587640] boot 2024-09-17T08:20:19.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:19 smithi047 ceph-mon[25866]: osdmap e8: 1 total, 1 up, 1 in 2024-09-17T08:20:19.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:19 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:20:19.939 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:20:20.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:20 smithi193 ceph-mon[30002]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:20.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:20 smithi193 ceph-mon[30002]: osdmap e9: 1 total, 1 up, 1 in 2024-09-17T08:20:20.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:20 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:20.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:20 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:20.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:20 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:20:20.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:20 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:20.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:20 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:20.645 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:20 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:20:20.645 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:20 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:20.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:20 smithi203 ceph-mon[30259]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:20.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:20 smithi203 ceph-mon[30259]: osdmap e9: 1 total, 1 up, 1 in 2024-09-17T08:20:20.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:20 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:20.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:20 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:20.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:20 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:20:20.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:20 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:20.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:20 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:20.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:20 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:20:20.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:20 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:20.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:20 smithi047 ceph-mon[25866]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:20.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:20 smithi047 ceph-mon[25866]: osdmap e9: 1 total, 1 up, 1 in 2024-09-17T08:20:20.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:20 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:20.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:20 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:20.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:20 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:20:20.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:20 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:20.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:20 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:20.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:20 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:20:20.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:20 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:21.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:21 smithi203 ceph-mon[30259]: Detected new or changed devices on smithi047 2024-09-17T08:20:21.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:21 smithi203 ceph-mon[30259]: Adjusting osd_memory_target on smithi047 to 17092M 2024-09-17T08:20:21.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:21 smithi047 ceph-mon[25866]: Detected new or changed devices on smithi047 2024-09-17T08:20:21.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:21 smithi047 ceph-mon[25866]: Adjusting osd_memory_target on smithi047 to 17092M 2024-09-17T08:20:21.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:21 smithi193 ceph-mon[30002]: Detected new or changed devices on smithi047 2024-09-17T08:20:21.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:21 smithi193 ceph-mon[30002]: Adjusting osd_memory_target on smithi047 to 17092M 2024-09-17T08:20:22.618 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:22 smithi193 ceph-mon[30002]: from='client.24128 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:20:22.618 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:22 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:20:22.618 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:22 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:20:22.618 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:22 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:22.618 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:22 smithi193 ceph-mon[30002]: pgmap v46: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:22.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:22 smithi203 ceph-mon[30259]: from='client.24128 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:20:22.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:22 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:20:22.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:22 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:20:22.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:22 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:22.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:22 smithi203 ceph-mon[30259]: pgmap v46: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:22.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:22 smithi047 ceph-mon[25866]: from='client.24128 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:20:22.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:22 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:20:22.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:22 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:20:22.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:22 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:22.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:22 smithi047 ceph-mon[25866]: pgmap v46: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:23.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:23 smithi193 ceph-mon[30002]: pgmap v47: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:24.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:23 smithi203 ceph-mon[30259]: pgmap v47: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:24.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:23 smithi047 ceph-mon[25866]: pgmap v47: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:25.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:24 smithi203 ceph-mon[30259]: from='client.? 172.21.15.193:0/1821858925' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93d2b7ff-defc-4d83-a5d9-ac8d12dc4606"}]: dispatch 2024-09-17T08:20:25.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:24 smithi203 ceph-mon[30259]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93d2b7ff-defc-4d83-a5d9-ac8d12dc4606"}]: dispatch 2024-09-17T08:20:25.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:24 smithi203 ceph-mon[30259]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93d2b7ff-defc-4d83-a5d9-ac8d12dc4606"}]': finished 2024-09-17T08:20:25.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:24 smithi203 ceph-mon[30259]: osdmap e10: 2 total, 1 up, 2 in 2024-09-17T08:20:25.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:24 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:20:25.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:24 smithi047 ceph-mon[25866]: from='client.? 172.21.15.193:0/1821858925' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93d2b7ff-defc-4d83-a5d9-ac8d12dc4606"}]: dispatch 2024-09-17T08:20:25.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:24 smithi047 ceph-mon[25866]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93d2b7ff-defc-4d83-a5d9-ac8d12dc4606"}]: dispatch 2024-09-17T08:20:25.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:24 smithi047 ceph-mon[25866]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93d2b7ff-defc-4d83-a5d9-ac8d12dc4606"}]': finished 2024-09-17T08:20:25.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:24 smithi047 ceph-mon[25866]: osdmap e10: 2 total, 1 up, 2 in 2024-09-17T08:20:25.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:24 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:20:25.093 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:24 smithi193 ceph-mon[30002]: from='client.? 172.21.15.193:0/1821858925' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93d2b7ff-defc-4d83-a5d9-ac8d12dc4606"}]: dispatch 2024-09-17T08:20:25.093 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:24 smithi193 ceph-mon[30002]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93d2b7ff-defc-4d83-a5d9-ac8d12dc4606"}]: dispatch 2024-09-17T08:20:25.093 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:24 smithi193 ceph-mon[30002]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93d2b7ff-defc-4d83-a5d9-ac8d12dc4606"}]': finished 2024-09-17T08:20:25.093 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:24 smithi193 ceph-mon[30002]: osdmap e10: 2 total, 1 up, 2 in 2024-09-17T08:20:25.094 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:24 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:20:26.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:25 smithi203 ceph-mon[30259]: from='client.? 172.21.15.193:0/4196383853' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:20:26.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:25 smithi203 ceph-mon[30259]: pgmap v49: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:26.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:25 smithi047 ceph-mon[25866]: from='client.? 172.21.15.193:0/4196383853' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:20:26.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:25 smithi047 ceph-mon[25866]: pgmap v49: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:26.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:25 smithi193 ceph-mon[30002]: from='client.? 172.21.15.193:0/4196383853' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:20:26.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:25 smithi193 ceph-mon[30002]: pgmap v49: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:28.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:27 smithi203 ceph-mon[30259]: pgmap v50: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:28.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:27 smithi047 ceph-mon[25866]: pgmap v50: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:28.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:27 smithi193 ceph-mon[30002]: pgmap v50: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:29.990 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:29 smithi193 ceph-mon[30002]: pgmap v51: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:30.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:29 smithi203 ceph-mon[30259]: pgmap v51: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:30.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:29 smithi047 ceph-mon[25866]: pgmap v51: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:31.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:30 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-17T08:20:31.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:30 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:31.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:30 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-17T08:20:31.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:30 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:31.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:30 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-17T08:20:31.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:30 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:32.110 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:31 smithi193 ceph-mon[30002]: Deploying daemon osd.1 on smithi193 2024-09-17T08:20:32.110 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:31 smithi193 ceph-mon[30002]: pgmap v52: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:32.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:31 smithi203 ceph-mon[30259]: Deploying daemon osd.1 on smithi193 2024-09-17T08:20:32.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:31 smithi203 ceph-mon[30259]: pgmap v52: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:32.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:31 smithi047 ceph-mon[25866]: Deploying daemon osd.1 on smithi193 2024-09-17T08:20:32.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:31 smithi047 ceph-mon[25866]: pgmap v52: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:34.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:33 smithi203 ceph-mon[30259]: pgmap v53: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:34.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:33 smithi047 ceph-mon[25866]: pgmap v53: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:34.077 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:33 smithi193 ceph-mon[30002]: pgmap v53: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:36.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:35 smithi203 ceph-mon[30259]: pgmap v54: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:36.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:35 smithi047 ceph-mon[25866]: pgmap v54: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:36.098 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:35 smithi193 ceph-mon[30002]: pgmap v54: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:36.895 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:36 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:20:36.895 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:36 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:36.895 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:36 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:37.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:36 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:20:37.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:36 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:37.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:36 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:37.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:36 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:20:37.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:36 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:37.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:36 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:37.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:37 smithi193 ceph-mon[30002]: pgmap v55: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:38.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:37 smithi203 ceph-mon[30259]: pgmap v55: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:38.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:37 smithi047 ceph-mon[25866]: pgmap v55: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:39.035 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:39.036 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:39.036 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:39.036 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:20:39.036 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:39.036 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:38 smithi193 ceph-mon[30002]: from='osd.1 [v2:172.21.15.193:6800/602719163,v1:172.21.15.193:6801/602719163]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T08:20:39.036 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:38 smithi193 ceph-mon[30002]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T08:20:39.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:38 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:39.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:38 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:39.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:38 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:39.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:38 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:20:39.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:38 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:39.285 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:38 smithi203 ceph-mon[30259]: from='osd.1 [v2:172.21.15.193:6800/602719163,v1:172.21.15.193:6801/602719163]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T08:20:39.285 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:38 smithi203 ceph-mon[30259]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T08:20:39.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:38 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:39.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:38 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:39.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:38 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:39.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:38 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:20:39.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:38 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:39.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:38 smithi047 ceph-mon[25866]: from='osd.1 [v2:172.21.15.193:6800/602719163,v1:172.21.15.193:6801/602719163]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T08:20:39.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:38 smithi047 ceph-mon[25866]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T08:20:39.354 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 1 on host 'smithi193' 2024-09-17T08:20:39.919 DEBUG:teuthology.orchestra.run.smithi193:osd.1> sudo journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@osd.1.service 2024-09-17T08:20:39.922 INFO:tasks.cephadm:Deploying osd.2 on smithi203 with /dev/nvme1n1... 2024-09-17T08:20:39.922 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme1n1 2024-09-17T08:20:40.124 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T08:20:40.162 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:39 smithi193 ceph-mon[30002]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-17T08:20:40.162 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:39 smithi193 ceph-mon[30002]: osdmap e11: 2 total, 1 up, 2 in 2024-09-17T08:20:40.162 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:39 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:20:40.162 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:39 smithi193 ceph-mon[30002]: from='osd.1 [v2:172.21.15.193:6800/602719163,v1:172.21.15.193:6801/602719163]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-17T08:20:40.162 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:39 smithi193 ceph-mon[30002]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-17T08:20:40.162 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:39 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:20:40.162 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:39 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:40.163 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:39 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:40.163 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:39 smithi193 ceph-mon[30002]: pgmap v57: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:40.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:39 smithi203 ceph-mon[30259]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-17T08:20:40.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:39 smithi203 ceph-mon[30259]: osdmap e11: 2 total, 1 up, 2 in 2024-09-17T08:20:40.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:39 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:20:40.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:39 smithi203 ceph-mon[30259]: from='osd.1 [v2:172.21.15.193:6800/602719163,v1:172.21.15.193:6801/602719163]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-17T08:20:40.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:39 smithi203 ceph-mon[30259]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-17T08:20:40.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:39 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:20:40.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:39 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:40.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:39 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:40.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:39 smithi203 ceph-mon[30259]: pgmap v57: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:40.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:39 smithi047 ceph-mon[25866]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-17T08:20:40.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:39 smithi047 ceph-mon[25866]: osdmap e11: 2 total, 1 up, 2 in 2024-09-17T08:20:40.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:39 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:20:40.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:39 smithi047 ceph-mon[25866]: from='osd.1 [v2:172.21.15.193:6800/602719163,v1:172.21.15.193:6801/602719163]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-17T08:20:40.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:39 smithi047 ceph-mon[25866]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-17T08:20:40.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:39 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:20:40.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:39 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:40.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:39 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:40.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:39 smithi047 ceph-mon[25866]: pgmap v57: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:40.795 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:20:40 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-1[34409]: 2024-09-17T08:20:40.419+0000 7f573dbf5640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T08:20:40.796 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:20:40 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-1[34409]: 2024-09-17T08:20:40.422+0000 7f5735a16640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T08:20:41.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:40 smithi193 ceph-mon[30002]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-17T08:20:41.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:40 smithi193 ceph-mon[30002]: osdmap e12: 2 total, 1 up, 2 in 2024-09-17T08:20:41.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:40 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:20:41.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:40 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:20:41.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:40 smithi193 ceph-mon[30002]: from='osd.1 ' entity='osd.1' 2024-09-17T08:20:41.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:40 smithi203 ceph-mon[30259]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-17T08:20:41.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:40 smithi203 ceph-mon[30259]: osdmap e12: 2 total, 1 up, 2 in 2024-09-17T08:20:41.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:40 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:20:41.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:40 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:20:41.285 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:40 smithi203 ceph-mon[30259]: from='osd.1 ' entity='osd.1' 2024-09-17T08:20:41.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:40 smithi047 ceph-mon[25866]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-17T08:20:41.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:40 smithi047 ceph-mon[25866]: osdmap e12: 2 total, 1 up, 2 in 2024-09-17T08:20:41.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:40 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:20:41.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:40 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:20:41.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:40 smithi047 ceph-mon[25866]: from='osd.1 ' entity='osd.1' 2024-09-17T08:20:41.982 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/nvme1n1 2024-09-17T08:20:41.982 INFO:teuthology.orchestra.run.smithi203.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-17T08:20:41.986 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2024-09-17T08:20:42.001 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2024-09-17T08:20:42.001 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2024-09-17T08:20:42.001 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125206 s, 837 MB/s 2024-09-17T08:20:42.001 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2024-09-17T08:20:42.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:41 smithi203 ceph-mon[30259]: purged_snaps scrub starts 2024-09-17T08:20:42.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:41 smithi203 ceph-mon[30259]: purged_snaps scrub ok 2024-09-17T08:20:42.285 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:41 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:20:42.285 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:41 smithi203 ceph-mon[30259]: osd.1 [v2:172.21.15.193:6800/602719163,v1:172.21.15.193:6801/602719163] boot 2024-09-17T08:20:42.285 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:41 smithi203 ceph-mon[30259]: osdmap e13: 2 total, 2 up, 2 in 2024-09-17T08:20:42.285 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:41 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:20:42.285 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:41 smithi203 ceph-mon[30259]: pgmap v60: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:42.286 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:41 smithi193 ceph-mon[30002]: purged_snaps scrub starts 2024-09-17T08:20:42.286 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:41 smithi193 ceph-mon[30002]: purged_snaps scrub ok 2024-09-17T08:20:42.286 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:41 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:20:42.286 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:41 smithi193 ceph-mon[30002]: osd.1 [v2:172.21.15.193:6800/602719163,v1:172.21.15.193:6801/602719163] boot 2024-09-17T08:20:42.286 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:41 smithi193 ceph-mon[30002]: osdmap e13: 2 total, 2 up, 2 in 2024-09-17T08:20:42.286 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:41 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:20:42.287 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:41 smithi193 ceph-mon[30002]: pgmap v60: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:42.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:41 smithi047 ceph-mon[25866]: purged_snaps scrub starts 2024-09-17T08:20:42.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:41 smithi047 ceph-mon[25866]: purged_snaps scrub ok 2024-09-17T08:20:42.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:41 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:20:42.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:41 smithi047 ceph-mon[25866]: osd.1 [v2:172.21.15.193:6800/602719163,v1:172.21.15.193:6801/602719163] boot 2024-09-17T08:20:42.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:41 smithi047 ceph-mon[25866]: osdmap e13: 2 total, 2 up, 2 in 2024-09-17T08:20:42.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:41 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:20:42.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:41 smithi047 ceph-mon[25866]: pgmap v60: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:20:42.591 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi203:/dev/nvme1n1 2024-09-17T08:20:42.779 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T08:20:43.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:43 smithi203 ceph-mon[30259]: osdmap e14: 2 total, 2 up, 2 in 2024-09-17T08:20:43.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:43 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:43.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:43 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:43.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:43 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:20:43.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:43 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:43.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:43 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:43.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:43 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:20:43.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:43 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:43.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:43 smithi047 ceph-mon[25866]: osdmap e14: 2 total, 2 up, 2 in 2024-09-17T08:20:43.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:43 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:43.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:43 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:43.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:43 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:20:43.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:43 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:43.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:43 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:43.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:43 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:20:43.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:43 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:43.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:43 smithi193 ceph-mon[30002]: osdmap e14: 2 total, 2 up, 2 in 2024-09-17T08:20:43.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:43 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:43.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:43 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:43.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:43 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:20:43.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:43 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:43.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:43 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:43.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:43 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:20:43.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:43 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:44.691 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:44 smithi047 ceph-mon[25866]: Detected new or changed devices on smithi193 2024-09-17T08:20:44.691 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:44 smithi047 ceph-mon[25866]: Adjusting osd_memory_target on smithi193 to 17088M 2024-09-17T08:20:44.692 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:44 smithi047 ceph-mon[25866]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:44.780 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:44 smithi203 ceph-mon[30259]: Detected new or changed devices on smithi193 2024-09-17T08:20:44.780 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:44 smithi203 ceph-mon[30259]: Adjusting osd_memory_target on smithi193 to 17088M 2024-09-17T08:20:44.780 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:44 smithi203 ceph-mon[30259]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:44.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:44 smithi193 ceph-mon[30002]: Detected new or changed devices on smithi193 2024-09-17T08:20:44.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:44 smithi193 ceph-mon[30002]: Adjusting osd_memory_target on smithi193 to 17088M 2024-09-17T08:20:44.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:44 smithi193 ceph-mon[30002]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:45.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:45 smithi047 ceph-mon[25866]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:20:45.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:45 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:20:45.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:45 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:20:45.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:45 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:45.743 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:45 smithi203 ceph-mon[30259]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:20:45.743 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:45 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:20:45.743 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:45 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:20:45.744 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:45 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:45.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:45 smithi193 ceph-mon[30002]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:20:45.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:45 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:20:45.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:45 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:20:45.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:45 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:46.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:46 smithi193 ceph-mon[30002]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:46.775 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:46 smithi203 ceph-mon[30259]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:46.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:46 smithi047 ceph-mon[25866]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:47.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:47 smithi203 ceph-mon[30259]: from='client.? 172.21.15.203:0/3309637149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc13aeff-7651-4b41-8a7a-6577984dc735"}]: dispatch 2024-09-17T08:20:47.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:47 smithi203 ceph-mon[30259]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc13aeff-7651-4b41-8a7a-6577984dc735"}]: dispatch 2024-09-17T08:20:47.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:47 smithi203 ceph-mon[30259]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc13aeff-7651-4b41-8a7a-6577984dc735"}]': finished 2024-09-17T08:20:47.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:47 smithi203 ceph-mon[30259]: osdmap e15: 3 total, 2 up, 3 in 2024-09-17T08:20:47.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:47 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:20:47.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:47 smithi047 ceph-mon[25866]: from='client.? 172.21.15.203:0/3309637149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc13aeff-7651-4b41-8a7a-6577984dc735"}]: dispatch 2024-09-17T08:20:47.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:47 smithi047 ceph-mon[25866]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc13aeff-7651-4b41-8a7a-6577984dc735"}]: dispatch 2024-09-17T08:20:47.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:47 smithi047 ceph-mon[25866]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc13aeff-7651-4b41-8a7a-6577984dc735"}]': finished 2024-09-17T08:20:47.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:47 smithi047 ceph-mon[25866]: osdmap e15: 3 total, 2 up, 3 in 2024-09-17T08:20:47.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:47 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:20:47.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:47 smithi193 ceph-mon[30002]: from='client.? 172.21.15.203:0/3309637149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc13aeff-7651-4b41-8a7a-6577984dc735"}]: dispatch 2024-09-17T08:20:47.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:47 smithi193 ceph-mon[30002]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc13aeff-7651-4b41-8a7a-6577984dc735"}]: dispatch 2024-09-17T08:20:47.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:47 smithi193 ceph-mon[30002]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc13aeff-7651-4b41-8a7a-6577984dc735"}]': finished 2024-09-17T08:20:47.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:47 smithi193 ceph-mon[30002]: osdmap e15: 3 total, 2 up, 3 in 2024-09-17T08:20:47.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:47 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:20:48.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:48 smithi203 ceph-mon[30259]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:48.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:48 smithi203 ceph-mon[30259]: from='client.? 172.21.15.203:0/927973682' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:20:48.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:48 smithi047 ceph-mon[25866]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:48.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:48 smithi047 ceph-mon[25866]: from='client.? 172.21.15.203:0/927973682' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:20:48.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:48 smithi193 ceph-mon[30002]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:48.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:48 smithi193 ceph-mon[30002]: from='client.? 172.21.15.203:0/927973682' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:20:50.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:49 smithi203 ceph-mon[30259]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:50.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:49 smithi047 ceph-mon[25866]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:50.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:49 smithi193 ceph-mon[30002]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:52.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:51 smithi203 ceph-mon[30259]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:52.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:51 smithi047 ceph-mon[25866]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:52.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:51 smithi193 ceph-mon[30002]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:53.476 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:53 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-17T08:20:53.477 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:53 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:53.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:53 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-17T08:20:53.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:53 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:53.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:53 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-17T08:20:53.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:53 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:54.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:54 smithi203 ceph-mon[30259]: Deploying daemon osd.2 on smithi203 2024-09-17T08:20:54.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:54 smithi203 ceph-mon[30259]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:54.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:54 smithi047 ceph-mon[25866]: Deploying daemon osd.2 on smithi203 2024-09-17T08:20:54.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:54 smithi047 ceph-mon[25866]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:54.643 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:54 smithi193 ceph-mon[30002]: Deploying daemon osd.2 on smithi203 2024-09-17T08:20:54.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:54 smithi193 ceph-mon[30002]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:55.947 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:55 smithi203 ceph-mon[30259]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:56.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:55 smithi047 ceph-mon[25866]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:56.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:55 smithi193 ceph-mon[30002]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:58.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:57 smithi203 ceph-mon[30259]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:58.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:57 smithi047 ceph-mon[25866]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:58.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:57 smithi193 ceph-mon[30002]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:20:58.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:58 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:20:58.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:58 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:58.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:58 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:59.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:58 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:20:59.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:58 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:59.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:58 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:59.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:58 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:20:59.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:58 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:59.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:58 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:20:59.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:20:59 smithi203 ceph-mon[30259]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:21:00.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:20:59 smithi047 ceph-mon[25866]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:21:00.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:20:59 smithi193 ceph-mon[30002]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:21:00.795 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 2 on host 'smithi203' 2024-09-17T08:21:01.388 DEBUG:teuthology.orchestra.run.smithi203:osd.2> sudo journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@osd.2.service 2024-09-17T08:21:01.390 INFO:tasks.cephadm:Waiting for 3 OSDs to come up... 2024-09-17T08:21:01.390 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-17T08:21:01.601 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:01.984 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:01 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:01.984 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:01 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:01.984 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:01 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:01.984 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:01 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:01.984 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:01 smithi047 ceph-mon[25866]: from='osd.2 [v2:172.21.15.203:6800/3699903128,v1:172.21.15.203:6801/3699903128]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:21:01.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:01 smithi047 ceph-mon[25866]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:21:01.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:01 smithi047 ceph-mon[25866]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:21:02.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:01 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:02.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:01 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:02.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:01 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:02.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:01 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:02.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:01 smithi203 ceph-mon[30259]: from='osd.2 [v2:172.21.15.203:6800/3699903128,v1:172.21.15.203:6801/3699903128]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:21:02.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:01 smithi203 ceph-mon[30259]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:21:02.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:01 smithi203 ceph-mon[30259]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:21:02.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:01 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:02.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:01 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:02.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:01 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:02.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:01 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:02.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:01 smithi193 ceph-mon[30002]: from='osd.2 [v2:172.21.15.203:6800/3699903128,v1:172.21.15.203:6801/3699903128]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:21:02.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:01 smithi193 ceph-mon[30002]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:21:02.145 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:01 smithi193 ceph-mon[30002]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:21:03.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:03 smithi047 ceph-mon[25866]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-17T08:21:03.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:03 smithi047 ceph-mon[25866]: osdmap e16: 3 total, 2 up, 3 in 2024-09-17T08:21:03.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:03 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:21:03.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:03 smithi047 ceph-mon[25866]: from='osd.2 [v2:172.21.15.203:6800/3699903128,v1:172.21.15.203:6801/3699903128]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-17T08:21:03.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:03 smithi047 ceph-mon[25866]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-17T08:21:03.393 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:03 smithi193 ceph-mon[30002]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-17T08:21:03.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:03 smithi193 ceph-mon[30002]: osdmap e16: 3 total, 2 up, 3 in 2024-09-17T08:21:03.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:03 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:21:03.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:03 smithi193 ceph-mon[30002]: from='osd.2 [v2:172.21.15.203:6800/3699903128,v1:172.21.15.203:6801/3699903128]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-17T08:21:03.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:03 smithi193 ceph-mon[30002]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-17T08:21:03.482 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:21:03.529 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:03 smithi203 ceph-mon[30259]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-17T08:21:03.529 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:03 smithi203 ceph-mon[30259]: osdmap e16: 3 total, 2 up, 3 in 2024-09-17T08:21:03.530 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:03 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:21:03.530 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:03 smithi203 ceph-mon[30259]: from='osd.2 [v2:172.21.15.203:6800/3699903128,v1:172.21.15.203:6801/3699903128]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-17T08:21:03.530 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:03 smithi203 ceph-mon[30259]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-17T08:21:03.784 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:21:03 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-2[35366]: 2024-09-17T08:21:03.528+0000 7f16890d9640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T08:21:03.784 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:21:03 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-2[35366]: 2024-09-17T08:21:03.531+0000 7f1680efa640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T08:21:04.050 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"num_osds":3,"num_up_osds":2,"osd_up_since":1726561241,"num_in_osds":3,"osd_in_since":1726561246,"num_remapped_pgs":0} 2024-09-17T08:21:04.170 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:04 smithi203 ceph-mon[30259]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-17T08:21:04.171 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:04 smithi203 ceph-mon[30259]: osdmap e17: 3 total, 2 up, 3 in 2024-09-17T08:21:04.171 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:04 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:21:04.171 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:04 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:21:04.171 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:04 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/3545073477' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-17T08:21:04.171 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:04 smithi203 ceph-mon[30259]: from='osd.2 ' entity='osd.2' 2024-09-17T08:21:04.171 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:04 smithi203 ceph-mon[30259]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:21:04.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:04 smithi047 ceph-mon[25866]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-17T08:21:04.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:04 smithi047 ceph-mon[25866]: osdmap e17: 3 total, 2 up, 3 in 2024-09-17T08:21:04.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:04 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:21:04.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:04 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:21:04.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:04 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/3545073477' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-17T08:21:04.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:04 smithi047 ceph-mon[25866]: from='osd.2 ' entity='osd.2' 2024-09-17T08:21:04.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:04 smithi047 ceph-mon[25866]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:21:04.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:04 smithi193 ceph-mon[30002]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-17T08:21:04.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:04 smithi193 ceph-mon[30002]: osdmap e17: 3 total, 2 up, 3 in 2024-09-17T08:21:04.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:04 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:21:04.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:04 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:21:04.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:04 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/3545073477' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-17T08:21:04.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:04 smithi193 ceph-mon[30002]: from='osd.2 ' entity='osd.2' 2024-09-17T08:21:04.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:04 smithi193 ceph-mon[30002]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:21:05.052 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-17T08:21:05.253 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:05.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:05 smithi047 ceph-mon[25866]: purged_snaps scrub starts 2024-09-17T08:21:05.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:05 smithi047 ceph-mon[25866]: purged_snaps scrub ok 2024-09-17T08:21:05.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:05 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:21:05.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:05 smithi047 ceph-mon[25866]: osd.2 [v2:172.21.15.203:6800/3699903128,v1:172.21.15.203:6801/3699903128] boot 2024-09-17T08:21:05.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:05 smithi047 ceph-mon[25866]: osdmap e18: 3 total, 3 up, 3 in 2024-09-17T08:21:05.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:05 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:21:05.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:05 smithi047 ceph-mon[25866]: Detected new or changed devices on smithi203 2024-09-17T08:21:05.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:05 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:05.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:05 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:05.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:05 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:05.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:05 smithi047 ceph-mon[25866]: Adjusting osd_memory_target on smithi203 to 17088M 2024-09-17T08:21:05.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:05 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:05.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:05 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:21:05.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:05 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:21:05.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:05 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:05.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:05 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:21:05.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:05 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:21:05.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:05 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:21:05.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:05 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:05.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:05 smithi193 ceph-mon[30002]: purged_snaps scrub starts 2024-09-17T08:21:05.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:05 smithi193 ceph-mon[30002]: purged_snaps scrub ok 2024-09-17T08:21:05.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:05 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:21:05.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:05 smithi193 ceph-mon[30002]: osd.2 [v2:172.21.15.203:6800/3699903128,v1:172.21.15.203:6801/3699903128] boot 2024-09-17T08:21:05.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:05 smithi193 ceph-mon[30002]: osdmap e18: 3 total, 3 up, 3 in 2024-09-17T08:21:05.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:05 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:21:05.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:05 smithi193 ceph-mon[30002]: Detected new or changed devices on smithi203 2024-09-17T08:21:05.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:05 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:05.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:05 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:05.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:05 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:05.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:05 smithi193 ceph-mon[30002]: Adjusting osd_memory_target on smithi203 to 17088M 2024-09-17T08:21:05.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:05 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:05.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:05 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:21:05.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:05 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:21:05.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:05 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:05.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:05 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:21:05.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:05 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:21:05.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:05 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:21:05.395 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:05 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:05.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:05 smithi203 ceph-mon[30259]: purged_snaps scrub starts 2024-09-17T08:21:05.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:05 smithi203 ceph-mon[30259]: purged_snaps scrub ok 2024-09-17T08:21:05.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:05 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:21:05.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:05 smithi203 ceph-mon[30259]: osd.2 [v2:172.21.15.203:6800/3699903128,v1:172.21.15.203:6801/3699903128] boot 2024-09-17T08:21:05.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:05 smithi203 ceph-mon[30259]: osdmap e18: 3 total, 3 up, 3 in 2024-09-17T08:21:05.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:05 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:21:05.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:05 smithi203 ceph-mon[30259]: Detected new or changed devices on smithi203 2024-09-17T08:21:05.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:05 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:05.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:05 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:05.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:05 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:05.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:05 smithi203 ceph-mon[30259]: Adjusting osd_memory_target on smithi203 to 17088M 2024-09-17T08:21:05.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:05 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:05.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:05 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:21:05.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:05 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:21:05.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:05 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:05.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:05 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:21:05.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:05 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:21:05.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:05 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:21:05.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:05 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:06.427 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:06 smithi047 ceph-mon[25866]: pgmap v77: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:06.428 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:06 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:21:06.428 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:06 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-17T08:21:06.428 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:06 smithi047 ceph-mon[25866]: osdmap e19: 3 total, 3 up, 3 in 2024-09-17T08:21:06.428 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:06 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:21:06.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:06 smithi203 ceph-mon[30259]: pgmap v77: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:06.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:06 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:21:06.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:06 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-17T08:21:06.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:06 smithi203 ceph-mon[30259]: osdmap e19: 3 total, 3 up, 3 in 2024-09-17T08:21:06.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:06 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:21:06.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:06 smithi193 ceph-mon[30002]: pgmap v77: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:06.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:06 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:21:06.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:06 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-17T08:21:06.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:06 smithi193 ceph-mon[30002]: osdmap e19: 3 total, 3 up, 3 in 2024-09-17T08:21:06.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:06 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:21:07.083 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:21:07.429 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:07 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/2417683666' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-17T08:21:07.429 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:07 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:21:07.429 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:07 smithi047 ceph-mon[25866]: osdmap e20: 3 total, 3 up, 3 in 2024-09-17T08:21:07.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:07 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/2417683666' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-17T08:21:07.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:07 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:21:07.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:07 smithi203 ceph-mon[30259]: osdmap e20: 3 total, 3 up, 3 in 2024-09-17T08:21:07.643 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:07 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/2417683666' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-17T08:21:07.643 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:07 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:21:07.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:07 smithi193 ceph-mon[30002]: osdmap e20: 3 total, 3 up, 3 in 2024-09-17T08:21:07.690 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:21:07 smithi047 sudo[49524]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1 2024-09-17T08:21:07.690 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:21:07 smithi047 sudo[49524]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:21:07.690 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:21:07 smithi047 sudo[49524]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:21:07.690 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:21:07 smithi047 sudo[49524]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:21:07.691 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":19,"num_osds":3,"num_up_osds":3,"osd_up_since":1726561264,"num_in_osds":3,"osd_in_since":1726561246,"num_remapped_pgs":0} 2024-09-17T08:21:07.692 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-17T08:21:07.892 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:08.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:08 smithi047 ceph-mon[25866]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:08.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:08 smithi047 ceph-mon[25866]: osdmap e21: 3 total, 3 up, 3 in 2024-09-17T08:21:08.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:08 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:08.290 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:21:08 smithi047 sudo[49600]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme1n1 2024-09-17T08:21:08.290 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:21:08 smithi047 sudo[49600]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:21:08.290 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:21:08 smithi047 sudo[49600]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:21:08.290 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:21:08 smithi047 sudo[49600]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:21:08.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:08 smithi203 ceph-mon[30259]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:08.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:08 smithi203 ceph-mon[30259]: osdmap e21: 3 total, 3 up, 3 in 2024-09-17T08:21:08.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:08 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:08.627 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:08 smithi193 ceph-mon[30002]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:08.627 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:08 smithi193 ceph-mon[30002]: osdmap e21: 3 total, 3 up, 3 in 2024-09-17T08:21:08.627 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:08 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:08.628 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:21:08 smithi193 sudo[39814]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1 2024-09-17T08:21:08.628 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:21:08 smithi193 sudo[39814]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:21:08.628 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:21:08 smithi193 sudo[39814]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:21:08.628 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:21:08 smithi193 sudo[39814]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:21:08.894 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:21:08 smithi193 sudo[39817]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme1n1 2024-09-17T08:21:08.894 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:21:08 smithi193 sudo[39817]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:21:08.894 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:21:08 smithi193 sudo[39817]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:21:08.894 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:21:08 smithi193 sudo[39817]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:21:09.150 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:21:08 smithi203 sudo[40937]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1 2024-09-17T08:21:09.150 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:21:08 smithi203 sudo[40937]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:21:09.150 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:21:08 smithi203 sudo[40937]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:21:09.150 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:21:08 smithi203 sudo[40937]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:21:09.280 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:09 smithi047 ceph-mon[25866]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-09-17T08:21:09.280 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:09 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:09.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:09 smithi203 ceph-mon[30259]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-09-17T08:21:09.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:09 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:09.535 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:21:09 smithi203 sudo[40940]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme1n1 2024-09-17T08:21:09.535 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:21:09 smithi203 sudo[40940]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:21:09.535 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:21:09 smithi203 sudo[40940]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:21:09.535 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:21:09 smithi203 sudo[40940]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:21:09.643 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:09 smithi193 ceph-mon[30002]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-09-17T08:21:09.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:09 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:10.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:09 smithi047 sudo[49713]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T08:21:10.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:09 smithi047 sudo[49713]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:21:10.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:09 smithi047 sudo[49713]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:21:10.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:09 smithi047 sudo[49713]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:21:10.262 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:21:10.262 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"3a66518a-74cd-11ef-bceb-c7b262605968","created":"2024-09-17T08:17:56.595672+0000","modified":"2024-09-17T08:21:08.112828+0000","last_up_change":"2024-09-17T08:21:04.530692+0000","last_in_change":"2024-09-17T08:20:46.721602+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T08:21:06.110129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ec4352f-9153-4631-9033-a6203fe48eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":619587640},{"type":"v1","addr":"172.21.15.47:6803","nonce":619587640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":619587640},{"type":"v1","addr":"172.21.15.47:6805","nonce":619587640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":619587640},{"type":"v1","addr":"172.21.15.47:6809","nonce":619587640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":619587640},{"type":"v1","addr":"172.21.15.47:6807","nonce":619587640}]},"public_addr":"172.21.15.47:6803/619587640","cluster_addr":"172.21.15.47:6805/619587640","heartbeat_back_addr":"172.21.15.47:6809/619587640","heartbeat_front_addr":"172.21.15.47:6807/619587640","state":["exists","up"]},{"osd":1,"uuid":"93d2b7ff-defc-4d83-a5d9-ac8d12dc4606","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":602719163},{"type":"v1","addr":"172.21.15.193:6801","nonce":602719163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":602719163},{"type":"v1","addr":"172.21.15.193:6803","nonce":602719163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":602719163},{"type":"v1","addr":"172.21.15.193:6807","nonce":602719163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":602719163},{"type":"v1","addr":"172.21.15.193:6805","nonce":602719163}]},"public_addr":"172.21.15.193:6801/602719163","cluster_addr":"172.21.15.193:6803/602719163","heartbeat_back_addr":"172.21.15.193:6807/602719163","heartbeat_front_addr":"172.21.15.193:6805/602719163","state":["exists","up"]},{"osd":2,"uuid":"cc13aeff-7651-4b41-8a7a-6577984dc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3699903128},{"type":"v1","addr":"172.21.15.203:6801","nonce":3699903128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3699903128},{"type":"v1","addr":"172.21.15.203:6803","nonce":3699903128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3699903128},{"type":"v1","addr":"172.21.15.203:6807","nonce":3699903128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3699903128},{"type":"v1","addr":"172.21.15.203:6805","nonce":3699903128}]},"public_addr":"172.21.15.203:6801/3699903128","cluster_addr":"172.21.15.203:6803/3699903128","heartbeat_back_addr":"172.21.15.203:6807/3699903128","heartbeat_front_addr":"172.21.15.203:6805/3699903128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:20:17.145469+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:20:39.789686+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:21:02.042358+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/3164287439":"2024-09-18T08:18:47.614683+0000","172.21.15.47:0/2549771690":"2024-09-18T08:18:47.614683+0000","172.21.15.47:6801/1596937753":"2024-09-18T08:18:47.614683+0000","172.21.15.47:6800/1596937753":"2024-09-18T08:18:47.614683+0000","172.21.15.47:0/1299440841":"2024-09-18T08:18:18.732865+0000","172.21.15.47:0/3657325753":"2024-09-18T08:18:18.732865+0000","172.21.15.47:6801/4136357482":"2024-09-18T08:18:18.732865+0000","172.21.15.47:0/632990711":"2024-09-18T08:18:47.614683+0000","172.21.15.47:6800/4136357482":"2024-09-18T08:18:18.732865+0000","172.21.15.47:0/3073557535":"2024-09-18T08:18:18.732865+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T08:21:10.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:10 smithi047 sudo[49716]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T08:21:10.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:10 smithi047 sudo[49716]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:21:10.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:10 smithi047 sudo[49716]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:21:10.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:10 smithi047 sudo[49716]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:21:10.597 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:10 smithi203 sudo[40943]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T08:21:10.598 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:10 smithi203 sudo[40943]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:21:10.598 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:10 smithi203 sudo[40943]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:21:10.598 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:10 smithi203 sudo[40943]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:21:10.819 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:10 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:10.819 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:10 smithi047 ceph-mon[25866]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:21:10.819 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:10 smithi047 ceph-mon[25866]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:10.819 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:10 smithi047 ceph-mon[25866]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:21:10.819 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:10 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:21:10.819 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:10 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:21:10.819 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:10 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:21:10.819 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:10 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:10.819 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:10 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:21:10.820 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:10 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:21:10.820 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:10 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:21:10.837 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T08:21:06.110129+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 3, 'score_stable': 3, 'optimal_score': 1, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T08:21:10.838 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-17T08:21:10.861 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:10 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:10.861 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:10 smithi193 ceph-mon[30002]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:21:10.862 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:10 smithi193 ceph-mon[30002]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:10.862 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:10 smithi193 ceph-mon[30002]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:21:10.862 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:10 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:21:10.862 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:10 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:21:10.862 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:10 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:21:10.862 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:10 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:10.862 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:10 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:21:10.862 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:10 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:21:10.862 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:10 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:21:11.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:10 smithi203 sudo[40946]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T08:21:11.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:10 smithi203 sudo[40946]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:21:11.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:10 smithi203 sudo[40946]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:21:11.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:10 smithi203 sudo[40946]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:21:11.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:10 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:11.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:10 smithi203 ceph-mon[30259]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:21:11.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:10 smithi203 ceph-mon[30259]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:11.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:10 smithi203 ceph-mon[30259]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:21:11.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:10 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:21:11.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:10 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:21:11.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:10 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:21:11.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:10 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:11.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:10 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:21:11.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:10 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:21:11.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:10 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:21:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:11.115 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:10 smithi193 sudo[39820]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T08:21:11.115 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:10 smithi193 sudo[39820]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:21:11.115 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:10 smithi193 sudo[39820]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:21:11.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:10 smithi193 sudo[39820]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:21:11.393 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:11 smithi193 sudo[39823]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T08:21:11.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:11 smithi193 sudo[39823]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:21:11.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:11 smithi193 sudo[39823]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:21:11.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:11 smithi193 sudo[39823]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:21:11.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:11 smithi047 ceph-mon[25866]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:21:11.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:11 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/1296524852' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:21:11.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:11 smithi047 ceph-mon[25866]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:21:11.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:11 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:11.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:11 smithi047 ceph-mon[25866]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:21:11.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:11 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:21:11.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:11 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:21:11.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:11 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:21:11.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:11 smithi047 ceph-mon[25866]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:21:11.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:11 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:11.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:11 smithi193 ceph-mon[30002]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:21:11.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:11 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/1296524852' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:21:11.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:11 smithi193 ceph-mon[30002]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:21:11.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:11 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:11.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:11 smithi193 ceph-mon[30002]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:21:11.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:11 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:21:11.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:11 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:21:11.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:11 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:21:11.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:11 smithi193 ceph-mon[30002]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:21:11.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:11 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:12.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:11 smithi203 ceph-mon[30259]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:21:12.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:11 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/1296524852' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:21:12.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:11 smithi203 ceph-mon[30259]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:21:12.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:11 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:12.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:11 smithi203 ceph-mon[30259]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:21:12.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:11 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:21:12.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:11 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:21:12.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:11 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:21:12.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:11 smithi203 ceph-mon[30259]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:21:12.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:11 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:12.783 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-09-17T08:21:12.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:12 smithi047 ceph-mon[25866]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:12.897 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:12 smithi193 ceph-mon[30002]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:13.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:12 smithi203 ceph-mon[30259]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:13.343 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi047 2024-09-17T08:21:13.343 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi047=foo.a' 2024-09-17T08:21:13.548 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T08:21:13.668 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:13 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/1987853110' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-17T08:21:13.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:13 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/1987853110' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-17T08:21:14.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:13 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/1987853110' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-17T08:21:14.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:14 smithi203 ceph-mon[30259]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:14.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:14 smithi193 ceph-mon[30002]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:15.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:14 smithi047 ceph-mon[25866]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:15.264 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled rgw.foo.a update... 2024-09-17T08:21:15.770 DEBUG:teuthology.orchestra.run.smithi047:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@rgw.foo.a.service 2024-09-17T08:21:15.772 INFO:tasks.cephadm:Adding prometheus.a on smithi193 2024-09-17T08:21:15.773 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph orch apply prometheus '1;smithi193=a' 2024-09-17T08:21:15.961 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T08:21:16.344 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:16 smithi203 ceph-mon[30259]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi047=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:21:16.344 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:16 smithi203 ceph-mon[30259]: Saving service rgw.foo.a spec with placement smithi047=foo.a;count:1 2024-09-17T08:21:16.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:16 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:16.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:16 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:21:16.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:16 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:21:16.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:16 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:21:16.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:16 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:16.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:16 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-17T08:21:16.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:16 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-17T08:21:16.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:16 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:16.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:16 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:21:16.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:16 smithi203 ceph-mon[30259]: Deploying daemon rgw.foo.a on smithi047 2024-09-17T08:21:16.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:16 smithi203 ceph-mon[30259]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:16.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:16 smithi047 ceph-mon[25866]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi047=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:21:16.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:16 smithi047 ceph-mon[25866]: Saving service rgw.foo.a spec with placement smithi047=foo.a;count:1 2024-09-17T08:21:16.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:16 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:16.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:16 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:21:16.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:16 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:21:16.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:16 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:21:16.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:16 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:16.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:16 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-17T08:21:16.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:16 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-17T08:21:16.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:16 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:16.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:16 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:21:16.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:16 smithi047 ceph-mon[25866]: Deploying daemon rgw.foo.a on smithi047 2024-09-17T08:21:16.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:16 smithi047 ceph-mon[25866]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:16.643 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:16 smithi193 ceph-mon[30002]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi047=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:21:16.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:16 smithi193 ceph-mon[30002]: Saving service rgw.foo.a spec with placement smithi047=foo.a;count:1 2024-09-17T08:21:16.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:16 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:16.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:16 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:21:16.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:16 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:21:16.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:16 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:21:16.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:16 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:16.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:16 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-17T08:21:16.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:16 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-17T08:21:16.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:16 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:16.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:16 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:21:16.645 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:16 smithi193 ceph-mon[30002]: Deploying daemon rgw.foo.a on smithi047 2024-09-17T08:21:16.645 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:16 smithi193 ceph-mon[30002]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:17.695 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled prometheus update... 2024-09-17T08:21:17.735 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 17 08:21:17 smithi047 podman[50263]: 2024-09-17 08:21:17.603259714 +0000 UTC m=+0.139710493 container create f02521794540c647642c45178861d87603137270f908470ce9c304feea3a91f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:21:17.986 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:17 smithi203 ceph-mon[30259]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:17.987 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:17 smithi203 ceph-mon[30259]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi193=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:21:17.987 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:17 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:18.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:17 smithi047 ceph-mon[25866]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:18.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:17 smithi047 ceph-mon[25866]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi193=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:21:18.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:17 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:18.040 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 17 08:21:17 smithi047 podman[50263]: 2024-09-17 08:21:17.856084657 +0000 UTC m=+0.392535434 container init f02521794540c647642c45178861d87603137270f908470ce9c304feea3a91f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-17T08:21:18.040 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 17 08:21:17 smithi047 podman[50263]: 2024-09-17 08:21:17.861219469 +0000 UTC m=+0.397670246 container start f02521794540c647642c45178861d87603137270f908470ce9c304feea3a91f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:21:18.040 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 17 08:21:17 smithi047 bash[50263]: f02521794540c647642c45178861d87603137270f908470ce9c304feea3a91f2 2024-09-17T08:21:18.040 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 17 08:21:17 smithi047 systemd[1]: Started Ceph rgw.foo.a for 3a66518a-74cd-11ef-bceb-c7b262605968. 2024-09-17T08:21:18.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:17 smithi193 ceph-mon[30002]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:18.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:17 smithi193 ceph-mon[30002]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi193=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:21:18.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:17 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:18.274 DEBUG:teuthology.orchestra.run.smithi193:prometheus.a> sudo journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@prometheus.a.service 2024-09-17T08:21:18.277 INFO:tasks.cephadm:Adding node-exporter.a on smithi047 2024-09-17T08:21:18.277 INFO:tasks.cephadm:Adding node-exporter.b on smithi193 2024-09-17T08:21:18.277 INFO:tasks.cephadm:Adding node-exporter.c on smithi203 2024-09-17T08:21:18.277 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph orch apply node-exporter '3;smithi047=a;smithi193=b;smithi203=c' 2024-09-17T08:21:18.470 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T08:21:19.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:19 smithi203 ceph-mon[30259]: Saving service prometheus spec with placement smithi193=a;count:1 2024-09-17T08:21:19.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:19 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:19.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:19 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:19.534 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:19 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:19.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:19 smithi203 ceph-mon[30259]: Saving service rgw.foo.a spec with placement smithi047=foo.a;count:1 2024-09-17T08:21:19.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:19 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:19.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:19 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:19.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:19 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:21:19.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:19 smithi203 ceph-mon[30259]: osdmap e22: 3 total, 3 up, 3 in 2024-09-17T08:21:19.535 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:19 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/3621054899' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T08:21:19.643 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:19 smithi193 ceph-mon[30002]: Saving service prometheus spec with placement smithi193=a;count:1 2024-09-17T08:21:19.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:19 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:19.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:19 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:19.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:19 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:19.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:19 smithi193 ceph-mon[30002]: Saving service rgw.foo.a spec with placement smithi047=foo.a;count:1 2024-09-17T08:21:19.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:19 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:19.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:19 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:19.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:19 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:21:19.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:19 smithi193 ceph-mon[30002]: osdmap e22: 3 total, 3 up, 3 in 2024-09-17T08:21:19.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:19 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/3621054899' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T08:21:19.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:19 smithi047 ceph-mon[25866]: Saving service prometheus spec with placement smithi193=a;count:1 2024-09-17T08:21:19.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:19 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:19.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:19 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:19.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:19 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:19.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:19 smithi047 ceph-mon[25866]: Saving service rgw.foo.a spec with placement smithi047=foo.a;count:1 2024-09-17T08:21:19.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:19 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:19.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:19 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:19.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:19 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:21:19.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:19 smithi047 ceph-mon[25866]: osdmap e22: 3 total, 3 up, 3 in 2024-09-17T08:21:19.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:19 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/3621054899' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T08:21:20.210 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled node-exporter update... 2024-09-17T08:21:20.786 DEBUG:teuthology.orchestra.run.smithi047:node-exporter.a> sudo journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@node-exporter.a.service 2024-09-17T08:21:20.788 DEBUG:teuthology.orchestra.run.smithi193:node-exporter.b> sudo journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@node-exporter.b.service 2024-09-17T08:21:20.791 DEBUG:teuthology.orchestra.run.smithi203:node-exporter.c> sudo journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@node-exporter.c.service 2024-09-17T08:21:20.793 INFO:tasks.cephadm:Adding alertmanager.a on smithi047 2024-09-17T08:21:20.793 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph orch apply alertmanager '1;smithi047=a' 2024-09-17T08:21:20.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:20 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:20.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:20 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:20.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:20 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:21:20.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:20 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:21:20.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:20 smithi203 ceph-mon[30259]: Checking dashboard <-> RGW credentials 2024-09-17T08:21:20.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:20 smithi203 ceph-mon[30259]: pgmap v88: 33 pgs: 8 creating+peering, 24 unknown, 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:20.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:20 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/3621054899' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-17T08:21:20.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:20 smithi203 ceph-mon[30259]: osdmap e23: 3 total, 3 up, 3 in 2024-09-17T08:21:20.814 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:20 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:20.997 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T08:21:21.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:20 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:21.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:20 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:21.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:20 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:21:21.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:20 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:21:21.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:20 smithi047 ceph-mon[25866]: Checking dashboard <-> RGW credentials 2024-09-17T08:21:21.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:20 smithi047 ceph-mon[25866]: pgmap v88: 33 pgs: 8 creating+peering, 24 unknown, 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:21.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:20 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/3621054899' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-17T08:21:21.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:20 smithi047 ceph-mon[25866]: osdmap e23: 3 total, 3 up, 3 in 2024-09-17T08:21:21.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:20 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:21.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:20 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:21.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:20 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:21.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:20 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:21:21.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:20 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:21:21.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:20 smithi193 ceph-mon[30002]: Checking dashboard <-> RGW credentials 2024-09-17T08:21:21.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:20 smithi193 ceph-mon[30002]: pgmap v88: 33 pgs: 8 creating+peering, 24 unknown, 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:21.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:20 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/3621054899' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-17T08:21:21.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:20 smithi193 ceph-mon[30002]: osdmap e23: 3 total, 3 up, 3 in 2024-09-17T08:21:21.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:20 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:21.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:21 smithi203 ceph-mon[30259]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi047=a;smithi193=b;smithi203=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:21:21.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:21 smithi203 ceph-mon[30259]: Saving service node-exporter spec with placement smithi047=a;smithi193=b;smithi203=c;count:3 2024-09-17T08:21:21.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:21 smithi203 ceph-mon[30259]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:21:21.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:21 smithi203 ceph-mon[30259]: osdmap e24: 3 total, 3 up, 3 in 2024-09-17T08:21:21.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:21 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/3621054899' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:21:21.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:21 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/261289531' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:21:22.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:21 smithi047 ceph-mon[25866]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi047=a;smithi193=b;smithi203=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:21:22.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:21 smithi047 ceph-mon[25866]: Saving service node-exporter spec with placement smithi047=a;smithi193=b;smithi203=c;count:3 2024-09-17T08:21:22.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:21 smithi047 ceph-mon[25866]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:21:22.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:21 smithi047 ceph-mon[25866]: osdmap e24: 3 total, 3 up, 3 in 2024-09-17T08:21:22.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:21 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/3621054899' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:21:22.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:21 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/261289531' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:21:22.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:21 smithi193 ceph-mon[30002]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi047=a;smithi193=b;smithi203=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:21:22.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:21 smithi193 ceph-mon[30002]: Saving service node-exporter spec with placement smithi047=a;smithi193=b;smithi203=c;count:3 2024-09-17T08:21:22.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:21 smithi193 ceph-mon[30002]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:21:22.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:21 smithi193 ceph-mon[30002]: osdmap e24: 3 total, 3 up, 3 in 2024-09-17T08:21:22.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:21 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/3621054899' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:21:22.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:21 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/261289531' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:21:22.633 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled alertmanager update... 2024-09-17T08:21:22.943 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:22 smithi203 ceph-mon[30259]: pgmap v91: 65 pgs: 8 creating+peering, 42 unknown, 15 active+clean; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 682 B/s rd, 682 B/s wr, 1 op/s 2024-09-17T08:21:22.944 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:22 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/3621054899' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T08:21:22.944 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:22 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/261289531' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T08:21:22.944 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:22 smithi203 ceph-mon[30259]: osdmap e25: 3 total, 3 up, 3 in 2024-09-17T08:21:22.944 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:22 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:22.944 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:22 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:23.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:22 smithi047 ceph-mon[25866]: pgmap v91: 65 pgs: 8 creating+peering, 42 unknown, 15 active+clean; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 682 B/s rd, 682 B/s wr, 1 op/s 2024-09-17T08:21:23.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:22 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/3621054899' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T08:21:23.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:22 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/261289531' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T08:21:23.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:22 smithi047 ceph-mon[25866]: osdmap e25: 3 total, 3 up, 3 in 2024-09-17T08:21:23.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:22 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:23.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:22 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:23.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:22 smithi193 ceph-mon[30002]: pgmap v91: 65 pgs: 8 creating+peering, 42 unknown, 15 active+clean; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 682 B/s rd, 682 B/s wr, 1 op/s 2024-09-17T08:21:23.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:22 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/3621054899' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T08:21:23.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:22 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/261289531' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T08:21:23.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:22 smithi193 ceph-mon[30002]: osdmap e25: 3 total, 3 up, 3 in 2024-09-17T08:21:23.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:22 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:23.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:22 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:23.223 DEBUG:teuthology.orchestra.run.smithi047:alertmanager.a> sudo journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@alertmanager.a.service 2024-09-17T08:21:23.226 INFO:tasks.cephadm:Adding grafana.a on smithi193 2024-09-17T08:21:23.226 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph orch apply grafana '1;smithi193=a' 2024-09-17T08:21:23.413 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T08:21:23.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:23 smithi203 ceph-mon[30259]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi047=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:21:23.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:23 smithi203 ceph-mon[30259]: Saving service alertmanager spec with placement smithi047=a;count:1 2024-09-17T08:21:23.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:23 smithi203 ceph-mon[30259]: osdmap e26: 3 total, 3 up, 3 in 2024-09-17T08:21:23.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:23 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/3621054899' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:21:23.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:23 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/261289531' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:21:24.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:23 smithi047 ceph-mon[25866]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi047=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:21:24.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:23 smithi047 ceph-mon[25866]: Saving service alertmanager spec with placement smithi047=a;count:1 2024-09-17T08:21:24.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:23 smithi047 ceph-mon[25866]: osdmap e26: 3 total, 3 up, 3 in 2024-09-17T08:21:24.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:23 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/3621054899' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:21:24.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:23 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/261289531' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:21:24.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:23 smithi193 ceph-mon[30002]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi047=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:21:24.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:23 smithi193 ceph-mon[30002]: Saving service alertmanager spec with placement smithi047=a;count:1 2024-09-17T08:21:24.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:23 smithi193 ceph-mon[30002]: osdmap e26: 3 total, 3 up, 3 in 2024-09-17T08:21:24.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:23 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/3621054899' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:21:24.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:23 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/261289531' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:21:24.716 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:24 smithi203 ceph-mon[30259]: pgmap v94: 97 pgs: 8 creating+peering, 53 unknown, 36 active+clean; 578 KiB data, 93 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-09-17T08:21:24.716 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:24 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/3621054899' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T08:21:24.716 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:24 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/261289531' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T08:21:24.716 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:24 smithi203 ceph-mon[30259]: osdmap e27: 3 total, 3 up, 3 in 2024-09-17T08:21:25.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:24 smithi047 ceph-mon[25866]: pgmap v94: 97 pgs: 8 creating+peering, 53 unknown, 36 active+clean; 578 KiB data, 93 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-09-17T08:21:25.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:24 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/3621054899' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T08:21:25.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:24 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/261289531' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T08:21:25.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:24 smithi047 ceph-mon[25866]: osdmap e27: 3 total, 3 up, 3 in 2024-09-17T08:21:25.120 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled grafana update... 2024-09-17T08:21:25.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:24 smithi193 ceph-mon[30002]: pgmap v94: 97 pgs: 8 creating+peering, 53 unknown, 36 active+clean; 578 KiB data, 93 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-09-17T08:21:25.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:24 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/3621054899' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T08:21:25.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:24 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/261289531' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T08:21:25.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:24 smithi193 ceph-mon[30002]: osdmap e27: 3 total, 3 up, 3 in 2024-09-17T08:21:25.702 DEBUG:teuthology.orchestra.run.smithi193:grafana.a> sudo journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@grafana.a.service 2024-09-17T08:21:25.705 INFO:tasks.cephadm:Setting up client nodes... 2024-09-17T08:21:25.705 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-17T08:21:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:26.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:25 smithi203 ceph-mon[30259]: osdmap e28: 3 total, 3 up, 3 in 2024-09-17T08:21:26.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:25 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/107369005' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:21:26.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:25 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/33197882' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:21:26.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:25 smithi203 ceph-mon[30259]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:21:26.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:25 smithi203 ceph-mon[30259]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi193=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:21:26.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:25 smithi203 ceph-mon[30259]: Saving service grafana spec with placement smithi193=a;count:1 2024-09-17T08:21:26.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:25 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:26.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:25 smithi203 ceph-mon[30259]: pgmap v97: 129 pgs: 9 creating+peering, 32 unknown, 88 active+clean; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T08:21:26.035 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:25 smithi047 ceph-mon[25866]: osdmap e28: 3 total, 3 up, 3 in 2024-09-17T08:21:26.035 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:25 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/107369005' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:21:26.036 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:25 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/33197882' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:21:26.036 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:25 smithi047 ceph-mon[25866]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:21:26.036 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:25 smithi047 ceph-mon[25866]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi193=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:21:26.036 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:25 smithi047 ceph-mon[25866]: Saving service grafana spec with placement smithi193=a;count:1 2024-09-17T08:21:26.036 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:25 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:26.036 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:25 smithi047 ceph-mon[25866]: pgmap v97: 129 pgs: 9 creating+peering, 32 unknown, 88 active+clean; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T08:21:26.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:25 smithi193 ceph-mon[30002]: osdmap e28: 3 total, 3 up, 3 in 2024-09-17T08:21:26.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:25 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/107369005' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:21:26.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:25 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/33197882' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:21:26.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:25 smithi193 ceph-mon[30002]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:21:26.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:25 smithi193 ceph-mon[30002]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi193=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:21:26.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:25 smithi193 ceph-mon[30002]: Saving service grafana spec with placement smithi193=a;count:1 2024-09-17T08:21:26.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:25 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:26.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:25 smithi193 ceph-mon[30002]: pgmap v97: 129 pgs: 9 creating+peering, 32 unknown, 88 active+clean; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T08:21:27.015 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:26 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/107369005' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T08:21:27.015 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:26 smithi047 ceph-mon[25866]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T08:21:27.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:26 smithi047 ceph-mon[25866]: osdmap e29: 3 total, 3 up, 3 in 2024-09-17T08:21:27.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:26 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/33197882' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:21:27.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:26 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/107369005' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:21:27.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:26 smithi047 ceph-mon[25866]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:21:27.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:26 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/107369005' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T08:21:27.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:26 smithi203 ceph-mon[30259]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T08:21:27.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:26 smithi203 ceph-mon[30259]: osdmap e29: 3 total, 3 up, 3 in 2024-09-17T08:21:27.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:26 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/33197882' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:21:27.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:26 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/107369005' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:21:27.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:26 smithi203 ceph-mon[30259]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:21:27.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:26 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/107369005' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T08:21:27.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:26 smithi193 ceph-mon[30002]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T08:21:27.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:26 smithi193 ceph-mon[30002]: osdmap e29: 3 total, 3 up, 3 in 2024-09-17T08:21:27.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:26 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/33197882' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:21:27.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:26 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/107369005' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:21:27.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:26 smithi193 ceph-mon[30002]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:21:27.783 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-09-17T08:21:27.783 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAHPOlmSj1wLhAA33+Z/BYqJTli86YYfiAX0w== 2024-09-17T08:21:28.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:27 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/107369005' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T08:21:28.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:27 smithi203 ceph-mon[30259]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T08:21:28.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:27 smithi203 ceph-mon[30259]: osdmap e30: 3 total, 3 up, 3 in 2024-09-17T08:21:28.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:27 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:28.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:27 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:28.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:27 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:28.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:27 smithi203 ceph-mon[30259]: Deploying daemon prometheus.a on smithi193 2024-09-17T08:21:28.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:27 smithi203 ceph-mon[30259]: pgmap v100: 129 pgs: 9 creating+peering, 10 unknown, 110 active+clean; 579 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 26 KiB/s rd, 3.0 KiB/s wr, 62 op/s 2024-09-17T08:21:28.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:27 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/387303168' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:21:28.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:27 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/107369005' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T08:21:28.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:27 smithi047 ceph-mon[25866]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T08:21:28.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:27 smithi047 ceph-mon[25866]: osdmap e30: 3 total, 3 up, 3 in 2024-09-17T08:21:28.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:27 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:27 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:27 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:27 smithi047 ceph-mon[25866]: Deploying daemon prometheus.a on smithi193 2024-09-17T08:21:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:27 smithi047 ceph-mon[25866]: pgmap v100: 129 pgs: 9 creating+peering, 10 unknown, 110 active+clean; 579 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 26 KiB/s rd, 3.0 KiB/s wr, 62 op/s 2024-09-17T08:21:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:27 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/387303168' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:21:28.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:27 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/107369005' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T08:21:28.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:27 smithi193 ceph-mon[30002]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T08:21:28.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:27 smithi193 ceph-mon[30002]: osdmap e30: 3 total, 3 up, 3 in 2024-09-17T08:21:28.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:27 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:28.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:27 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:28.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:27 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:28.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:27 smithi193 ceph-mon[30002]: Deploying daemon prometheus.a on smithi193 2024-09-17T08:21:28.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:27 smithi193 ceph-mon[30002]: pgmap v100: 129 pgs: 9 creating+peering, 10 unknown, 110 active+clean; 579 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 26 KiB/s rd, 3.0 KiB/s wr, 62 op/s 2024-09-17T08:21:28.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:27 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/387303168' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:21:28.303 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-17T08:21:28.303 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-17T08:21:28.303 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T08:21:28.343 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-17T08:21:28.573 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:21:29.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:28 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/387303168' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T08:21:29.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:28 smithi047 ceph-mon[25866]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:21:29.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:28 smithi047 ceph-mon[25866]: Cluster is now healthy 2024-09-17T08:21:29.107 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:28 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/387303168' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T08:21:29.107 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:28 smithi203 ceph-mon[30259]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:21:29.107 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:28 smithi203 ceph-mon[30259]: Cluster is now healthy 2024-09-17T08:21:29.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:28 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/387303168' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T08:21:29.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:28 smithi193 ceph-mon[30002]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:21:29.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:28 smithi193 ceph-mon[30002]: Cluster is now healthy 2024-09-17T08:21:29.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:29 smithi193 ceph-mon[30002]: pgmap v101: 129 pgs: 9 creating+peering, 120 active+clean; 581 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 4.1 KiB/s wr, 110 op/s 2024-09-17T08:21:30.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:29 smithi203 ceph-mon[30259]: pgmap v101: 129 pgs: 9 creating+peering, 120 active+clean; 581 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 4.1 KiB/s wr, 110 op/s 2024-09-17T08:21:30.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:29 smithi047 ceph-mon[25866]: pgmap v101: 129 pgs: 9 creating+peering, 120 active+clean; 581 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 4.1 KiB/s wr, 110 op/s 2024-09-17T08:21:31.448 INFO:teuthology.orchestra.run.smithi193.stdout:[client.1] 2024-09-17T08:21:31.448 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQALPOlmBBprGhAAuAo2eJYRFr4f2Y0hArXjLA== 2024-09-17T08:21:31.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:31 smithi193 ceph-mon[30002]: from='client.? 172.21.15.193:0/2609200913' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:21:31.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:31 smithi193 ceph-mon[30002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:21:31.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:31 smithi193 ceph-mon[30002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T08:21:31.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:31 smithi203 ceph-mon[30259]: from='client.? 172.21.15.193:0/2609200913' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:21:31.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:31 smithi203 ceph-mon[30259]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:21:31.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:31 smithi203 ceph-mon[30259]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T08:21:31.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:31 smithi047 ceph-mon[25866]: from='client.? 172.21.15.193:0/2609200913' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:21:31.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:31 smithi047 ceph-mon[25866]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:21:31.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:31 smithi047 ceph-mon[25866]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T08:21:32.643 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:32 smithi193 ceph-mon[30002]: pgmap v102: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 58 KiB/s rd, 5.1 KiB/s wr, 139 op/s 2024-09-17T08:21:32.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:32 smithi203 ceph-mon[30259]: pgmap v102: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 58 KiB/s rd, 5.1 KiB/s wr, 139 op/s 2024-09-17T08:21:32.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:32 smithi047 ceph-mon[25866]: pgmap v102: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 58 KiB/s rd, 5.1 KiB/s wr, 139 op/s 2024-09-17T08:21:32.803 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-17T08:21:32.803 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-17T08:21:32.803 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-17T08:21:32.840 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.2 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-17T08:21:33.048 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T08:21:34.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:33 smithi203 ceph-mon[30259]: pgmap v103: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 57 KiB/s rd, 4.4 KiB/s wr, 139 op/s 2024-09-17T08:21:34.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:33 smithi047 ceph-mon[25866]: pgmap v103: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 57 KiB/s rd, 4.4 KiB/s wr, 139 op/s 2024-09-17T08:21:34.044 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:33 smithi193 ceph-mon[30002]: pgmap v103: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 57 KiB/s rd, 4.4 KiB/s wr, 139 op/s 2024-09-17T08:21:34.814 INFO:teuthology.orchestra.run.smithi203.stdout:[client.2] 2024-09-17T08:21:34.815 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQAOPOlm8/NHMBAA+awsajWzX/bIdXHG8EZCuQ== 2024-09-17T08:21:35.109 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:34 smithi203 ceph-mon[30259]: from='client.? 172.21.15.203:0/3992378200' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:21:35.109 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:34 smithi203 ceph-mon[30259]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:21:35.109 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:34 smithi203 ceph-mon[30259]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T08:21:35.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:34 smithi193 ceph-mon[30002]: from='client.? 172.21.15.203:0/3992378200' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:21:35.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:34 smithi193 ceph-mon[30002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:21:35.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:34 smithi193 ceph-mon[30002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T08:21:35.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:34 smithi047 ceph-mon[25866]: from='client.? 172.21.15.203:0/3992378200' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:21:35.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:34 smithi047 ceph-mon[25866]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:21:35.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:34 smithi047 ceph-mon[25866]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T08:21:35.388 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-17T08:21:35.389 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.client.2.keyring 2024-09-17T08:21:35.389 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-17T08:21:35.426 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T08:21:35.426 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-17T08:21:35.427 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-17T08:21:35.637 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:36.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:35 smithi047 ceph-mon[25866]: pgmap v104: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 113 op/s 2024-09-17T08:21:36.122 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:35 smithi193 ceph-mon[30002]: pgmap v104: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 113 op/s 2024-09-17T08:21:36.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:35 smithi203 ceph-mon[30259]: pgmap v104: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 113 op/s 2024-09-17T08:21:37.393 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:21:37.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:37 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/2011141550' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-17T08:21:37.643 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:37 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/2011141550' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-17T08:21:37.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:37 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/2011141550' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-17T08:21:38.004 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":16,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":3872338602},{"type":"v1","addr":"172.21.15.47:6801","nonce":3872338602}]},"active_addr":"172.21.15.47:6801/3872338602","active_change":"2024-09-17T08:18:47.614789+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24107,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.47:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":3855001647}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":1299094794}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":2168552824}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":2834266557}]}]} 2024-09-17T08:21:38.011 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-17T08:21:38.011 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-17T08:21:38.012 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-17T08:21:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:38.582 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 systemd[1]: Starting Ceph prometheus.a for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:21:38.848 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-mon[30002]: pgmap v105: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 42 KiB/s rd, 3.2 KiB/s wr, 102 op/s 2024-09-17T08:21:38.848 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:38.848 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 podman[40816]: 2024-09-17 08:21:38.485127071 +0000 UTC m=+0.016898855 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-17T08:21:38.849 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 podman[40816]: 2024-09-17 08:21:38.60217278 +0000 UTC m=+0.133944562 container create 1ba33cd85096b236e70eacb887a100da559613e99ee0f4494244e93b9c8dd0a7 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:21:38.849 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 podman[40816]: 2024-09-17 08:21:38.819210798 +0000 UTC m=+0.350982586 container init 1ba33cd85096b236e70eacb887a100da559613e99ee0f4494244e93b9c8dd0a7 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:21:38.849 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 podman[40816]: 2024-09-17 08:21:38.824162798 +0000 UTC m=+0.355934583 container start 1ba33cd85096b236e70eacb887a100da559613e99ee0f4494244e93b9c8dd0a7 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:21:38.849 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:21:38.848Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-17T08:21:38.849 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:21:38.848Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-17T08:21:38.849 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:21:38.848Z caller=main.go:570 level=info host_details="(Linux 5.14.0-508.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024 x86_64 smithi193 (none))" 2024-09-17T08:21:38.849 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:21:38.848Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-17T08:21:38.849 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:21:38.848Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-17T08:21:39.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:38 smithi203 ceph-mon[30259]: pgmap v105: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 42 KiB/s rd, 3.2 KiB/s wr, 102 op/s 2024-09-17T08:21:39.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:38 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:39.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:38 smithi047 ceph-mon[25866]: pgmap v105: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 42 KiB/s rd, 3.2 KiB/s wr, 102 op/s 2024-09-17T08:21:39.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:38 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:39.144 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:21:38.849Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-17T08:21:39.144 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:21:38.850Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-17T08:21:39.144 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:21:38.851Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-17T08:21:39.144 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:21:38.851Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-17T08:21:39.144 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:21:38.853Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-17T08:21:39.144 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:21:38.853Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.445µs 2024-09-17T08:21:39.144 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:21:38.853Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-17T08:21:39.144 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:21:38.853Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-17T08:21:39.145 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:21:38.853Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=24.398µs wal_replay_duration=187.037µs wbl_replay_duration=130ns total_replay_duration=237.104µs 2024-09-17T08:21:39.145 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:21:38.854Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-17T08:21:39.145 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:21:38.854Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-17T08:21:39.145 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:21:38.854Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-17T08:21:39.145 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:21:38.874Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=20.278994ms db_storage=708ns remote_storage=1.109µs web_handler=361ns query_engine=556ns scrape=391.883µs scrape_sd=98.424µs notify=714ns notify_sd=2.89µs rules=19.495562ms tracing=5.848µs 2024-09-17T08:21:39.145 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:21:38.874Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-17T08:21:39.145 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:21:38.874Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-17T08:21:39.145 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 bash[40816]: 1ba33cd85096b236e70eacb887a100da559613e99ee0f4494244e93b9c8dd0a7 2024-09-17T08:21:39.145 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:21:38 smithi193 systemd[1]: Started Ceph prometheus.a for 3a66518a-74cd-11ef-bceb-c7b262605968. 2024-09-17T08:21:40.012 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:39 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:40.012 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:39 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:40.012 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:39 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:40.012 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:39 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-17T08:21:40.012 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:39 smithi047 ceph-mon[25866]: pgmap v106: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 30 KiB/s rd, 2.0 KiB/s wr, 72 op/s 2024-09-17T08:21:40.036 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:21:40.036 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":30,"fsid":"3a66518a-74cd-11ef-bceb-c7b262605968","created":"2024-09-17T08:17:56.595672+0000","modified":"2024-09-17T08:21:26.771956+0000","last_up_change":"2024-09-17T08:21:04.530692+0000","last_in_change":"2024-09-17T08:20:46.721602+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T08:21:06.110129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-17T08:21:18.333708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-09-17T08:21:20.205214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-09-17T08:21:22.198497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.meta","create_time":"2024-09-17T08:21:24.258995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ec4352f-9153-4631-9033-a6203fe48eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":619587640},{"type":"v1","addr":"172.21.15.47:6803","nonce":619587640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":619587640},{"type":"v1","addr":"172.21.15.47:6805","nonce":619587640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":619587640},{"type":"v1","addr":"172.21.15.47:6809","nonce":619587640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":619587640},{"type":"v1","addr":"172.21.15.47:6807","nonce":619587640}]},"public_addr":"172.21.15.47:6803/619587640","cluster_addr":"172.21.15.47:6805/619587640","heartbeat_back_addr":"172.21.15.47:6809/619587640","heartbeat_front_addr":"172.21.15.47:6807/619587640","state":["exists","up"]},{"osd":1,"uuid":"93d2b7ff-defc-4d83-a5d9-ac8d12dc4606","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":602719163},{"type":"v1","addr":"172.21.15.193:6801","nonce":602719163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":602719163},{"type":"v1","addr":"172.21.15.193:6803","nonce":602719163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":602719163},{"type":"v1","addr":"172.21.15.193:6807","nonce":602719163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":602719163},{"type":"v1","addr":"172.21.15.193:6805","nonce":602719163}]},"public_addr":"172.21.15.193:6801/602719163","cluster_addr":"172.21.15.193:6803/602719163","heartbeat_back_addr":"172.21.15.193:6807/602719163","heartbeat_front_addr":"172.21.15.193:6805/602719163","state":["exists","up"]},{"osd":2,"uuid":"cc13aeff-7651-4b41-8a7a-6577984dc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3699903128},{"type":"v1","addr":"172.21.15.203:6801","nonce":3699903128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3699903128},{"type":"v1","addr":"172.21.15.203:6803","nonce":3699903128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3699903128},{"type":"v1","addr":"172.21.15.203:6807","nonce":3699903128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3699903128},{"type":"v1","addr":"172.21.15.203:6805","nonce":3699903128}]},"public_addr":"172.21.15.203:6801/3699903128","cluster_addr":"172.21.15.203:6803/3699903128","heartbeat_back_addr":"172.21.15.203:6807/3699903128","heartbeat_front_addr":"172.21.15.203:6805/3699903128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:20:17.145469+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:20:39.789686+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:21:02.042358+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/3164287439":"2024-09-18T08:18:47.614683+0000","172.21.15.47:0/2549771690":"2024-09-18T08:18:47.614683+0000","172.21.15.47:6801/1596937753":"2024-09-18T08:18:47.614683+0000","172.21.15.47:6800/1596937753":"2024-09-18T08:18:47.614683+0000","172.21.15.47:0/1299440841":"2024-09-18T08:18:18.732865+0000","172.21.15.47:0/3657325753":"2024-09-18T08:18:18.732865+0000","172.21.15.47:6801/4136357482":"2024-09-18T08:18:18.732865+0000","172.21.15.47:0/632990711":"2024-09-18T08:18:47.614683+0000","172.21.15.47:6800/4136357482":"2024-09-18T08:18:18.732865+0000","172.21.15.47:0/3073557535":"2024-09-18T08:18:18.732865+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T08:21:40.284 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:40 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: ignoring --setuser ceph since I am not root 2024-09-17T08:21:40.284 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:40 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: ignoring --setgroup ceph since I am not root 2024-09-17T08:21:40.284 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:40 smithi203 ceph-mgr[32337]: -- 172.21.15.203:0/1825267825 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c776996d00 con 0x55c77696b000 2024-09-17T08:21:40.285 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:40 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:40.131+0000 7f240c7f4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:21:40.285 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:40 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:40.188+0000 7f240c7f4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:21:40.285 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:39 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:40.285 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:39 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:40.285 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:39 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:40.285 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:39 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-17T08:21:40.285 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:39 smithi203 ceph-mon[30259]: pgmap v106: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 30 KiB/s rd, 2.0 KiB/s wr, 72 op/s 2024-09-17T08:21:40.289 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:40 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: ignoring --setuser ceph since I am not root 2024-09-17T08:21:40.289 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:40 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: ignoring --setgroup ceph since I am not root 2024-09-17T08:21:40.289 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:40 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:40.133+0000 7f1922a36040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:21:40.290 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:40 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:40.191+0000 7f1922a36040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:21:40.333 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:39 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:40.333 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:39 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:40.333 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:39 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' 2024-09-17T08:21:40.333 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:39 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-17T08:21:40.334 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:39 smithi193 ceph-mon[30002]: pgmap v106: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 30 KiB/s rd, 2.0 KiB/s wr, 72 op/s 2024-09-17T08:21:40.334 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:40 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: ignoring --setuser ceph since I am not root 2024-09-17T08:21:40.334 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:40 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: ignoring --setgroup ceph since I am not root 2024-09-17T08:21:40.334 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:40 smithi193 ceph-mgr[31458]: -- 172.21.15.193:0/952316886 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cb01756d00 con 0x55cb0172a800 2024-09-17T08:21:40.334 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:40 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:40.129+0000 7fe073713040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:21:40.334 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:40 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:40.186+0000 7fe073713040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:21:40.334 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:40 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:40.332+0000 7fe073713040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:21:40.567 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:40 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:40.346+0000 7f1922a36040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:21:40.567 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:40 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:40.416+0000 7f1922a36040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:21:40.568 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:40 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:40.567+0000 7f1922a36040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:21:40.643 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:40 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:40.398+0000 7fe073713040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:21:40.643 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:40 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:40.541+0000 7fe073713040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:21:40.689 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:40 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:40.336+0000 7f240c7f4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:21:40.689 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:40 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:40.402+0000 7f240c7f4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:21:40.689 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:40 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:40.549+0000 7f240c7f4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:21:40.689 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:40 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:40.689+0000 7f240c7f4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:21:40.691 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-17T08:21:40.691 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-17T08:21:40.892 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:40.983 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:40 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:40.714+0000 7f1922a36040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:21:40.984 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:40 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:40.681+0000 7fe073713040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:21:41.238 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:40 smithi047 ceph-mon[25866]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-17T08:21:41.239 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:40 smithi047 ceph-mon[25866]: mgrmap e17: y(active, since 2m), standbys: z, x 2024-09-17T08:21:41.239 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:40 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/1928193392' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:21:41.239 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:41 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:41.048+0000 7f1922a36040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:21:41.239 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:41 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:41.113+0000 7f1922a36040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:21:41.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:40 smithi203 ceph-mon[30259]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-17T08:21:41.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:40 smithi203 ceph-mon[30259]: mgrmap e17: y(active, since 2m), standbys: z, x 2024-09-17T08:21:41.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:40 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/1928193392' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:21:41.240 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:40 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:40.996+0000 7f240c7f4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:21:41.241 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:41 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:41.057+0000 7f240c7f4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:21:41.241 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:41 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:41.239+0000 7f240c7f4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:21:41.393 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:40 smithi193 ceph-mon[30002]: from='mgr.14150 172.21.15.47:0/607480702' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-17T08:21:41.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:40 smithi193 ceph-mon[30002]: mgrmap e17: y(active, since 2m), standbys: z, x 2024-09-17T08:21:41.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:40 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/1928193392' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:21:41.394 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:40 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:40.985+0000 7fe073713040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:21:41.394 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:41 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:41.045+0000 7fe073713040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:21:41.394 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:41 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:41.224+0000 7fe073713040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:21:41.539 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:41 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:41.310+0000 7f1922a36040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:21:42.266 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:42 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:42.225+0000 7f1922a36040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:21:42.393 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:42 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:42.088+0000 7fe073713040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:21:42.394 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:42 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:42.153+0000 7fe073713040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:21:42.534 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:42 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:42.109+0000 7f240c7f4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:21:42.534 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:42 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:42.175+0000 7f240c7f4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:21:42.539 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:42 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:42.298+0000 7f1922a36040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:21:42.669 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:21:42.669 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":30,"fsid":"3a66518a-74cd-11ef-bceb-c7b262605968","created":"2024-09-17T08:17:56.595672+0000","modified":"2024-09-17T08:21:26.771956+0000","last_up_change":"2024-09-17T08:21:04.530692+0000","last_in_change":"2024-09-17T08:20:46.721602+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T08:21:06.110129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-17T08:21:18.333708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-09-17T08:21:20.205214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-09-17T08:21:22.198497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.meta","create_time":"2024-09-17T08:21:24.258995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ec4352f-9153-4631-9033-a6203fe48eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":619587640},{"type":"v1","addr":"172.21.15.47:6803","nonce":619587640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":619587640},{"type":"v1","addr":"172.21.15.47:6805","nonce":619587640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":619587640},{"type":"v1","addr":"172.21.15.47:6809","nonce":619587640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":619587640},{"type":"v1","addr":"172.21.15.47:6807","nonce":619587640}]},"public_addr":"172.21.15.47:6803/619587640","cluster_addr":"172.21.15.47:6805/619587640","heartbeat_back_addr":"172.21.15.47:6809/619587640","heartbeat_front_addr":"172.21.15.47:6807/619587640","state":["exists","up"]},{"osd":1,"uuid":"93d2b7ff-defc-4d83-a5d9-ac8d12dc4606","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":602719163},{"type":"v1","addr":"172.21.15.193:6801","nonce":602719163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":602719163},{"type":"v1","addr":"172.21.15.193:6803","nonce":602719163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":602719163},{"type":"v1","addr":"172.21.15.193:6807","nonce":602719163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":602719163},{"type":"v1","addr":"172.21.15.193:6805","nonce":602719163}]},"public_addr":"172.21.15.193:6801/602719163","cluster_addr":"172.21.15.193:6803/602719163","heartbeat_back_addr":"172.21.15.193:6807/602719163","heartbeat_front_addr":"172.21.15.193:6805/602719163","state":["exists","up"]},{"osd":2,"uuid":"cc13aeff-7651-4b41-8a7a-6577984dc735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":3699903128},{"type":"v1","addr":"172.21.15.203:6801","nonce":3699903128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":3699903128},{"type":"v1","addr":"172.21.15.203:6803","nonce":3699903128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":3699903128},{"type":"v1","addr":"172.21.15.203:6807","nonce":3699903128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":3699903128},{"type":"v1","addr":"172.21.15.203:6805","nonce":3699903128}]},"public_addr":"172.21.15.203:6801/3699903128","cluster_addr":"172.21.15.203:6803/3699903128","heartbeat_back_addr":"172.21.15.203:6807/3699903128","heartbeat_front_addr":"172.21.15.203:6805/3699903128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:20:17.145469+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:20:39.789686+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:21:02.042358+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/3164287439":"2024-09-18T08:18:47.614683+0000","172.21.15.47:0/2549771690":"2024-09-18T08:18:47.614683+0000","172.21.15.47:6801/1596937753":"2024-09-18T08:18:47.614683+0000","172.21.15.47:6800/1596937753":"2024-09-18T08:18:47.614683+0000","172.21.15.47:0/1299440841":"2024-09-18T08:18:18.732865+0000","172.21.15.47:0/3657325753":"2024-09-18T08:18:18.732865+0000","172.21.15.47:6801/4136357482":"2024-09-18T08:18:18.732865+0000","172.21.15.47:0/632990711":"2024-09-18T08:18:47.614683+0000","172.21.15.47:6800/4136357482":"2024-09-18T08:18:18.732865+0000","172.21.15.47:0/3073557535":"2024-09-18T08:18:18.732865+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T08:21:42.997 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:42 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/3458791712' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:21:43.000 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:42 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/3458791712' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:21:43.000 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:42 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:42.885+0000 7f240c7f4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:21:43.000 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:43 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:43.000+0000 7f240c7f4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:21:43.104 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:42 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/3458791712' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:21:43.104 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:42 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:42.853+0000 7fe073713040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:21:43.105 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:42 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:42.965+0000 7fe073713040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:21:43.265 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:43 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:43.057+0000 7f1922a36040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:21:43.265 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:43 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:43.176+0000 7f1922a36040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:21:43.284 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:43 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:43.141+0000 7f240c7f4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:21:43.284 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:43 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:43.202+0000 7f240c7f4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:21:43.288 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-17T08:21:43.288 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-17T08:21:43.289 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-17T08:21:43.375 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:43 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:43.103+0000 7fe073713040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:21:43.375 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:43 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:43.164+0000 7fe073713040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:21:43.375 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:43 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:43.308+0000 7fe073713040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:21:43.375 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:43 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:43.374+0000 7fe073713040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:21:43.514 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:43.540 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:43 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:43.328+0000 7f1922a36040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:21:43.540 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:43 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:43.395+0000 7f1922a36040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:21:43.632 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:43 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:43.349+0000 7f240c7f4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:21:43.633 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:43 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:43.417+0000 7f240c7f4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:21:43.633 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:43 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:43.477+0000 7f240c7f4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:21:43.633 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:43 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:43.632+0000 7f240c7f4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:21:43.643 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:43 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:43.433+0000 7fe073713040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:21:43.644 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:43 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:43.586+0000 7fe073713040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:21:43.875 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:43 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:43.601+0000 7f1922a36040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:21:43.875 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:43 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:43.674+0000 7f1922a36040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:21:43.875 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:43 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:43.741+0000 7f1922a36040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:21:43.906 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:43 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:43.698+0000 7f240c7f4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:21:43.906 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:43 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:43.765+0000 7f240c7f4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:21:43.906 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:43 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:43.847+0000 7f240c7f4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:21:43.906 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:43 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:43.906+0000 7f240c7f4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:21:43.914 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:43 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:43.649+0000 7fe073713040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:21:43.914 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:43 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:43.715+0000 7fe073713040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:21:43.914 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:43 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:43.798+0000 7fe073713040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:21:43.914 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:43 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:43.856+0000 7fe073713040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:21:43.914 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:43 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:43.913+0000 7fe073713040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:21:44.137 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:43 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:43.913+0000 7f1922a36040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:21:44.137 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:43 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:43.981+0000 7f1922a36040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:21:44.137 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:44 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:44.050+0000 7f1922a36040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:21:44.137 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:44 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:44.136+0000 7f1922a36040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:21:44.180 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:44.181 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:44.284 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:43 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:43.965+0000 7f240c7f4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:21:44.412 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:44 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:44.202+0000 7f1922a36040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:21:44.412 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:44 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:44.265+0000 7f1922a36040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:21:44.982 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:44 smithi047 ceph-mon[25866]: Standby manager daemon x restarted 2024-09-17T08:21:44.982 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:44 smithi047 ceph-mon[25866]: Standby manager daemon x started 2024-09-17T08:21:44.982 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:44 smithi047 ceph-mon[25866]: from='mgr.? 172.21.15.193:0/1222359737' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T08:21:44.982 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:44 smithi047 ceph-mon[25866]: from='mgr.? 172.21.15.193:0/1222359737' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:21:44.982 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:44 smithi047 ceph-mon[25866]: from='mgr.? 172.21.15.193:0/1222359737' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T08:21:44.983 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:44 smithi047 ceph-mon[25866]: from='mgr.? 172.21.15.193:0/1222359737' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:21:45.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:44 smithi203 ceph-mon[30259]: Standby manager daemon x restarted 2024-09-17T08:21:45.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:44 smithi203 ceph-mon[30259]: Standby manager daemon x started 2024-09-17T08:21:45.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:44 smithi203 ceph-mon[30259]: from='mgr.? 172.21.15.193:0/1222359737' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T08:21:45.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:44 smithi203 ceph-mon[30259]: from='mgr.? 172.21.15.193:0/1222359737' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:21:45.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:44 smithi203 ceph-mon[30259]: from='mgr.? 172.21.15.193:0/1222359737' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T08:21:45.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:44 smithi203 ceph-mon[30259]: from='mgr.? 172.21.15.193:0/1222359737' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:21:45.035 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:44 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:44.739+0000 7f240c7f4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:21:45.035 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:44 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: [17/Sep/2024:08:21:44] ENGINE Bus STARTING 2024-09-17T08:21:45.035 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:44 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: CherryPy Checker: 2024-09-17T08:21:45.035 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:44 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: The Application mounted at '' has an empty config. 2024-09-17T08:21:45.035 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:44 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: 2024-09-17T08:21:45.035 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:44 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: [17/Sep/2024:08:21:44] ENGINE Serving on http://:::9283 2024-09-17T08:21:45.036 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:21:44 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z[32295]: [17/Sep/2024:08:21:44] ENGINE Bus STARTED 2024-09-17T08:21:45.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:44 smithi193 ceph-mon[30002]: Standby manager daemon x restarted 2024-09-17T08:21:45.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:44 smithi193 ceph-mon[30002]: Standby manager daemon x started 2024-09-17T08:21:45.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:44 smithi193 ceph-mon[30002]: from='mgr.? 172.21.15.193:0/1222359737' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T08:21:45.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:44 smithi193 ceph-mon[30002]: from='mgr.? 172.21.15.193:0/1222359737' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:21:45.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:44 smithi193 ceph-mon[30002]: from='mgr.? 172.21.15.193:0/1222359737' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T08:21:45.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:44 smithi193 ceph-mon[30002]: from='mgr.? 172.21.15.193:0/1222359737' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:21:45.144 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:44 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:44.686+0000 7fe073713040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:21:45.144 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:44 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: [17/Sep/2024:08:21:44] ENGINE Bus STARTING 2024-09-17T08:21:45.144 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:44 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: CherryPy Checker: 2024-09-17T08:21:45.145 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:44 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: The Application mounted at '' has an empty config. 2024-09-17T08:21:45.145 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:44 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-17T08:21:45.145 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:44 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: [17/Sep/2024:08:21:44] ENGINE Serving on http://:::9283 2024-09-17T08:21:45.145 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:21:44 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x[31435]: [17/Sep/2024:08:21:44] ENGINE Bus STARTED 2024-09-17T08:21:45.289 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:45.111+0000 7f1922a36040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:21:45.290 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: [17/Sep/2024:08:21:45] ENGINE Bus STARTING 2024-09-17T08:21:45.290 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: CherryPy Checker: 2024-09-17T08:21:45.290 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: The Application mounted at '' has an empty config. 2024-09-17T08:21:45.290 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:45.736 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: [17/Sep/2024:08:21:45] ENGINE Serving on http://:::9283 2024-09-17T08:21:45.736 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: [17/Sep/2024:08:21:45] ENGINE Bus STARTED 2024-09-17T08:21:45.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: Standby manager daemon z restarted 2024-09-17T08:21:45.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: from='mgr.? 172.21.15.203:0/3628590198' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-17T08:21:45.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: Standby manager daemon z started 2024-09-17T08:21:45.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: mgrmap e18: y(active, since 2m), standbys: z, x 2024-09-17T08:21:45.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: from='mgr.? 172.21.15.203:0/3628590198' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:21:45.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: from='mgr.? 172.21.15.203:0/3628590198' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-17T08:21:45.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: from='mgr.? 172.21.15.203:0/3628590198' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:21:45.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: Active manager daemon y restarted 2024-09-17T08:21:45.895 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: Activating manager daemon y 2024-09-17T08:21:45.895 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: osdmap e31: 3 total, 3 up, 3 in 2024-09-17T08:21:45.895 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: mgrmap e19: y(active, starting, since 0.00935563s), standbys: x, z 2024-09-17T08:21:45.895 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:21:45.895 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:21:45.895 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:21:45.895 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:21:45.895 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T08:21:45.895 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-17T08:21:45.895 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:21:45.895 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:21:45.895 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:21:45.895 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:21:45.895 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:21:45.895 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:21:45.896 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: Manager daemon y is now available 2024-09-17T08:21:45.896 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:21:45.896 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:45.896 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:21:45.896 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:21:45.896 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:45 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:21:46.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: Standby manager daemon z restarted 2024-09-17T08:21:46.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: from='mgr.? 172.21.15.203:0/3628590198' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-17T08:21:46.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: Standby manager daemon z started 2024-09-17T08:21:46.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: mgrmap e18: y(active, since 2m), standbys: z, x 2024-09-17T08:21:46.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: from='mgr.? 172.21.15.203:0/3628590198' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:21:46.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: from='mgr.? 172.21.15.203:0/3628590198' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-17T08:21:46.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: from='mgr.? 172.21.15.203:0/3628590198' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:21:46.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: Active manager daemon y restarted 2024-09-17T08:21:46.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: Activating manager daemon y 2024-09-17T08:21:46.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: osdmap e31: 3 total, 3 up, 3 in 2024-09-17T08:21:46.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: mgrmap e19: y(active, starting, since 0.00935563s), standbys: x, z 2024-09-17T08:21:46.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:21:46.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:21:46.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:21:46.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:21:46.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T08:21:46.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-17T08:21:46.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:21:46.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:21:46.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:21:46.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:21:46.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:21:46.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:21:46.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: Manager daemon y is now available 2024-09-17T08:21:46.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:21:46.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:46.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:21:46.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:21:46.038 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:45 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:21:46.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: Standby manager daemon z restarted 2024-09-17T08:21:46.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: from='mgr.? 172.21.15.203:0/3628590198' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-17T08:21:46.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: Standby manager daemon z started 2024-09-17T08:21:46.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: mgrmap e18: y(active, since 2m), standbys: z, x 2024-09-17T08:21:46.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: from='mgr.? 172.21.15.203:0/3628590198' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:21:46.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: from='mgr.? 172.21.15.203:0/3628590198' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-17T08:21:46.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: from='mgr.? 172.21.15.203:0/3628590198' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:21:46.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: Active manager daemon y restarted 2024-09-17T08:21:46.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: Activating manager daemon y 2024-09-17T08:21:46.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: osdmap e31: 3 total, 3 up, 3 in 2024-09-17T08:21:46.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: mgrmap e19: y(active, starting, since 0.00935563s), standbys: x, z 2024-09-17T08:21:46.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:21:46.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:21:46.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:21:46.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:21:46.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T08:21:46.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-17T08:21:46.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:21:46.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:21:46.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:21:46.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:21:46.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:21:46.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:21:46.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: Manager daemon y is now available 2024-09-17T08:21:46.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:21:46.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:46.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:21:46.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:21:46.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:45 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:21:47.332 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-mon[25866]: mgrmap e20: y(active, since 1.0184s), standbys: x, z 2024-09-17T08:21:47.332 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-mon[25866]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:47.332 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:47.332 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:47.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:47.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:47.333 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:47.236+0000 7f187ff2f640 -1 log_channel(cephadm) log [ERR] : [17/Sep/2024:08:21:47] ENGINE Error in HTTPServer.serve 2024-09-17T08:21:47.333 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: Traceback (most recent call last): 2024-09-17T08:21:47.333 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:21:47.333 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: self._connections.run(self.expiration_interval) 2024-09-17T08:21:47.333 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:21:47.333 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: self._run(expiration_interval) 2024-09-17T08:21:47.334 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:21:47.334 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:21:47.334 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:21:47.334 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:21:47.334 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:21:47.334 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: s = self.context.wrap_socket( 2024-09-17T08:21:47.334 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:21:47.334 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: return self.sslsocket_class._create( 2024-09-17T08:21:47.334 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:21:47.334 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: self.do_handshake() 2024-09-17T08:21:47.334 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:21:47.334 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: self._sslobj.do_handshake() 2024-09-17T08:21:47.334 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:21:47.335 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:47 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:21:47.393 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:47 smithi193 ceph-mon[30002]: mgrmap e20: y(active, since 1.0184s), standbys: x, z 2024-09-17T08:21:47.393 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:47 smithi193 ceph-mon[30002]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:47.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:47 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:47.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:47 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:47.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:47 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:47.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:47 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:47.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:47 smithi203 ceph-mon[30259]: mgrmap e20: y(active, since 1.0184s), standbys: x, z 2024-09-17T08:21:47.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:47 smithi203 ceph-mon[30259]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:47.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:47 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:47.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:47 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:47.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:47 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:47.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:47 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:48.392 INFO:teuthology.orchestra.run.smithi047.stdout:77309411339 2024-09-17T08:21:48.392 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-17T08:21:48.515 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:48 smithi047 ceph-mon[25866]: [17/Sep/2024:08:21:47] ENGINE Bus STARTING 2024-09-17T08:21:48.515 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:48 smithi047 ceph-mon[25866]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:48.515 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:48 smithi047 ceph-mon[25866]: [17/Sep/2024:08:21:47] ENGINE Serving on https://172.21.15.47:7150 2024-09-17T08:21:48.515 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:48 smithi047 ceph-mon[25866]: [17/Sep/2024:08:21:47] ENGINE Error in HTTPServer.serve 2024-09-17T08:21:48.515 INFO:journalctl@ceph.mon.a.smithi047.stdout: Traceback (most recent call last): 2024-09-17T08:21:48.515 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:21:48.515 INFO:journalctl@ceph.mon.a.smithi047.stdout: self._connections.run(self.expiration_interval) 2024-09-17T08:21:48.515 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:21:48.515 INFO:journalctl@ceph.mon.a.smithi047.stdout: self._run(expiration_interval) 2024-09-17T08:21:48.516 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:21:48.516 INFO:journalctl@ceph.mon.a.smithi047.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:21:48.516 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:21:48.516 INFO:journalctl@ceph.mon.a.smithi047.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:21:48.516 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:21:48.516 INFO:journalctl@ceph.mon.a.smithi047.stdout: s = self.context.wrap_socket( 2024-09-17T08:21:48.516 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:21:48.516 INFO:journalctl@ceph.mon.a.smithi047.stdout: return self.sslsocket_class._create( 2024-09-17T08:21:48.516 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:21:48.516 INFO:journalctl@ceph.mon.a.smithi047.stdout: self.do_handshake() 2024-09-17T08:21:48.516 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:21:48.516 INFO:journalctl@ceph.mon.a.smithi047.stdout: self._sslobj.do_handshake() 2024-09-17T08:21:48.516 INFO:journalctl@ceph.mon.a.smithi047.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:21:48.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:48 smithi047 ceph-mon[25866]: [17/Sep/2024:08:21:47] ENGINE Serving on http://172.21.15.47:8765 2024-09-17T08:21:48.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:48 smithi047 ceph-mon[25866]: [17/Sep/2024:08:21:47] ENGINE Bus STARTED 2024-09-17T08:21:48.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:48 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:48.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:48 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:48.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:48 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:48.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:48 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:48.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:48 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:48.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:48 smithi047 ceph-mon[25866]: Adjusting osd_memory_target on smithi193 to 16064M 2024-09-17T08:21:48.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:48 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:48.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:48 smithi047 ceph-mon[25866]: mgrmap e21: y(active, since 2s), standbys: x, z 2024-09-17T08:21:48.643 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:48 smithi193 ceph-mon[30002]: [17/Sep/2024:08:21:47] ENGINE Bus STARTING 2024-09-17T08:21:48.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:48 smithi193 ceph-mon[30002]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:48.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:48 smithi193 ceph-mon[30002]: [17/Sep/2024:08:21:47] ENGINE Serving on https://172.21.15.47:7150 2024-09-17T08:21:48.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:48 smithi193 ceph-mon[30002]: [17/Sep/2024:08:21:47] ENGINE Error in HTTPServer.serve 2024-09-17T08:21:48.644 INFO:journalctl@ceph.mon.b.smithi193.stdout: Traceback (most recent call last): 2024-09-17T08:21:48.644 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:21:48.644 INFO:journalctl@ceph.mon.b.smithi193.stdout: self._connections.run(self.expiration_interval) 2024-09-17T08:21:48.644 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:21:48.644 INFO:journalctl@ceph.mon.b.smithi193.stdout: self._run(expiration_interval) 2024-09-17T08:21:48.645 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:21:48.645 INFO:journalctl@ceph.mon.b.smithi193.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:21:48.645 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:21:48.645 INFO:journalctl@ceph.mon.b.smithi193.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:21:48.645 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:21:48.645 INFO:journalctl@ceph.mon.b.smithi193.stdout: s = self.context.wrap_socket( 2024-09-17T08:21:48.645 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:21:48.645 INFO:journalctl@ceph.mon.b.smithi193.stdout: return self.sslsocket_class._create( 2024-09-17T08:21:48.645 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:21:48.646 INFO:journalctl@ceph.mon.b.smithi193.stdout: self.do_handshake() 2024-09-17T08:21:48.646 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:21:48.646 INFO:journalctl@ceph.mon.b.smithi193.stdout: self._sslobj.do_handshake() 2024-09-17T08:21:48.646 INFO:journalctl@ceph.mon.b.smithi193.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:21:48.646 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:48 smithi193 ceph-mon[30002]: [17/Sep/2024:08:21:47] ENGINE Serving on http://172.21.15.47:8765 2024-09-17T08:21:48.646 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:48 smithi193 ceph-mon[30002]: [17/Sep/2024:08:21:47] ENGINE Bus STARTED 2024-09-17T08:21:48.646 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:48 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:48.646 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:48 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:48.646 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:48 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:48.646 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:48 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:48.646 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:48 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:48.647 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:48 smithi193 ceph-mon[30002]: Adjusting osd_memory_target on smithi193 to 16064M 2024-09-17T08:21:48.647 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:48 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:48.647 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:48 smithi193 ceph-mon[30002]: mgrmap e21: y(active, since 2s), standbys: x, z 2024-09-17T08:21:48.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:48 smithi203 ceph-mon[30259]: [17/Sep/2024:08:21:47] ENGINE Bus STARTING 2024-09-17T08:21:48.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:48 smithi203 ceph-mon[30259]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:21:48.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:48 smithi203 ceph-mon[30259]: [17/Sep/2024:08:21:47] ENGINE Serving on https://172.21.15.47:7150 2024-09-17T08:21:48.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:48 smithi203 ceph-mon[30259]: [17/Sep/2024:08:21:47] ENGINE Error in HTTPServer.serve 2024-09-17T08:21:48.785 INFO:journalctl@ceph.mon.c.smithi203.stdout: Traceback (most recent call last): 2024-09-17T08:21:48.785 INFO:journalctl@ceph.mon.c.smithi203.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:21:48.785 INFO:journalctl@ceph.mon.c.smithi203.stdout: self._connections.run(self.expiration_interval) 2024-09-17T08:21:48.785 INFO:journalctl@ceph.mon.c.smithi203.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:21:48.785 INFO:journalctl@ceph.mon.c.smithi203.stdout: self._run(expiration_interval) 2024-09-17T08:21:48.785 INFO:journalctl@ceph.mon.c.smithi203.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:21:48.785 INFO:journalctl@ceph.mon.c.smithi203.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:21:48.785 INFO:journalctl@ceph.mon.c.smithi203.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:21:48.786 INFO:journalctl@ceph.mon.c.smithi203.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:21:48.786 INFO:journalctl@ceph.mon.c.smithi203.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:21:48.786 INFO:journalctl@ceph.mon.c.smithi203.stdout: s = self.context.wrap_socket( 2024-09-17T08:21:48.786 INFO:journalctl@ceph.mon.c.smithi203.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:21:48.786 INFO:journalctl@ceph.mon.c.smithi203.stdout: return self.sslsocket_class._create( 2024-09-17T08:21:48.786 INFO:journalctl@ceph.mon.c.smithi203.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:21:48.786 INFO:journalctl@ceph.mon.c.smithi203.stdout: self.do_handshake() 2024-09-17T08:21:48.786 INFO:journalctl@ceph.mon.c.smithi203.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:21:48.787 INFO:journalctl@ceph.mon.c.smithi203.stdout: self._sslobj.do_handshake() 2024-09-17T08:21:48.787 INFO:journalctl@ceph.mon.c.smithi203.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:21:48.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:48 smithi203 ceph-mon[30259]: [17/Sep/2024:08:21:47] ENGINE Serving on http://172.21.15.47:8765 2024-09-17T08:21:48.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:48 smithi203 ceph-mon[30259]: [17/Sep/2024:08:21:47] ENGINE Bus STARTED 2024-09-17T08:21:48.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:48 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:48.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:48 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:48.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:48 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:48.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:48 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:48.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:48 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:48.788 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:48 smithi203 ceph-mon[30259]: Adjusting osd_memory_target on smithi193 to 16064M 2024-09-17T08:21:48.788 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:48 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:48.788 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:48 smithi203 ceph-mon[30259]: mgrmap e21: y(active, since 2s), standbys: x, z 2024-09-17T08:21:48.808 INFO:teuthology.orchestra.run.smithi047.stdout:55834574864 2024-09-17T08:21:48.808 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-17T08:21:49.640 INFO:teuthology.orchestra.run.smithi047.stdout:34359738389 2024-09-17T08:21:49.640 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-17T08:21:49.748 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:50.607 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:50 smithi047 ceph-mon[25866]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 69 KiB/s rd, 0 B/s wr, 38 op/s 2024-09-17T08:21:50.607 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:50 smithi047 ceph-mon[25866]: mgrmap e22: y(active, since 4s), standbys: x, z 2024-09-17T08:21:50.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:50 smithi193 ceph-mon[30002]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 69 KiB/s rd, 0 B/s wr, 38 op/s 2024-09-17T08:21:50.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:50 smithi193 ceph-mon[30002]: mgrmap e22: y(active, since 4s), standbys: x, z 2024-09-17T08:21:50.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:50 smithi203 ceph-mon[30259]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 69 KiB/s rd, 0 B/s wr, 38 op/s 2024-09-17T08:21:50.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:50 smithi203 ceph-mon[30259]: mgrmap e22: y(active, since 4s), standbys: x, z 2024-09-17T08:21:51.525 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:52.643 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:52 smithi193 ceph-mon[30002]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 41 KiB/s rd, 0 B/s wr, 23 op/s 2024-09-17T08:21:52.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:52 smithi203 ceph-mon[30259]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 41 KiB/s rd, 0 B/s wr, 23 op/s 2024-09-17T08:21:52.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:52 smithi047 ceph-mon[25866]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 41 KiB/s rd, 0 B/s wr, 23 op/s 2024-09-17T08:21:53.301 INFO:teuthology.orchestra.run.smithi047.stdout:55834574864 2024-09-17T08:21:53.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:53 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:53.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:53 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:53.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:53 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/718799097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-17T08:21:53.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:53 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:53.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:53 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:53.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:53 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:53.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:53 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:53.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:53 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:21:53.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:53 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:21:53.787 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:53 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:53.787 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:53 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:53.787 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:53 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/718799097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-17T08:21:53.787 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:53 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:53.787 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:53 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:53.787 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:53 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:53.787 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:53 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:53.787 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:53 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:21:53.787 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:53 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:21:53.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:53 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:53.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:53 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:53.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:53 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/718799097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-17T08:21:53.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:53 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:53.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:53 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:53.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:53 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:53.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:53 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:53.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:53 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:21:53.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:53 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:21:54.162 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574864 got 55834574864 for osd.1 2024-09-17T08:21:54.163 DEBUG:teuthology.parallel:result is None 2024-09-17T08:21:54.598 INFO:teuthology.orchestra.run.smithi047.stdout:34359738390 2024-09-17T08:21:54.689 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:21:54 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: ::ffff:172.21.15.193 - - [17/Sep/2024:08:21:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:21:54.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:54 smithi047 ceph-mon[25866]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-17T08:21:54.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:54 smithi047 ceph-mon[25866]: Adjusting osd_memory_target on smithi047 to 16068M 2024-09-17T08:21:54.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:54 smithi047 ceph-mon[25866]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-17T08:21:54.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:54 smithi047 ceph-mon[25866]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-17T08:21:54.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:54 smithi047 ceph-mon[25866]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-17T08:21:54.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:54 smithi047 ceph-mon[25866]: Updating smithi047:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:21:54.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:54 smithi047 ceph-mon[25866]: Updating smithi193:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:21:54.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:54 smithi047 ceph-mon[25866]: Updating smithi203:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:21:54.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:54 smithi047 ceph-mon[25866]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:21:54.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:54 smithi047 ceph-mon[25866]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:21:54.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:54 smithi047 ceph-mon[25866]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:21:54.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:54 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:54.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:54 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:54.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:54 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:54.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:54 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:54.691 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:54 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:54.691 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:54 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:54.691 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:54 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:54.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:54 smithi203 ceph-mon[30259]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-17T08:21:54.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:54 smithi203 ceph-mon[30259]: Adjusting osd_memory_target on smithi047 to 16068M 2024-09-17T08:21:54.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:54 smithi203 ceph-mon[30259]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-17T08:21:54.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:54 smithi203 ceph-mon[30259]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-17T08:21:54.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:54 smithi203 ceph-mon[30259]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-17T08:21:54.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:54 smithi203 ceph-mon[30259]: Updating smithi047:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:21:54.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:54 smithi203 ceph-mon[30259]: Updating smithi193:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:21:54.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:54 smithi203 ceph-mon[30259]: Updating smithi203:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:21:54.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:54 smithi203 ceph-mon[30259]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:21:54.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:54 smithi203 ceph-mon[30259]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:21:54.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:54 smithi203 ceph-mon[30259]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:21:54.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:54 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:54.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:54 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:54.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:54 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:54.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:54 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:54.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:54 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:54.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:54 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:54.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:54 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:54.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:54 smithi193 ceph-mon[30002]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-17T08:21:54.896 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:54 smithi193 ceph-mon[30002]: Adjusting osd_memory_target on smithi047 to 16068M 2024-09-17T08:21:54.896 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:54 smithi193 ceph-mon[30002]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-17T08:21:54.896 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:54 smithi193 ceph-mon[30002]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-17T08:21:54.896 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:54 smithi193 ceph-mon[30002]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-17T08:21:54.896 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:54 smithi193 ceph-mon[30002]: Updating smithi047:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:21:54.896 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:54 smithi193 ceph-mon[30002]: Updating smithi193:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:21:54.896 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:54 smithi193 ceph-mon[30002]: Updating smithi203:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:21:54.896 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:54 smithi193 ceph-mon[30002]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:21:54.896 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:54 smithi193 ceph-mon[30002]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:21:54.896 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:54 smithi193 ceph-mon[30002]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:21:54.896 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:54 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:54.896 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:54 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:54.896 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:54 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:54.897 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:54 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:54.897 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:54 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:54.897 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:54 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:54.897 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:54 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:21:55.541 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738389 got 34359738390 for osd.0 2024-09-17T08:21:55.541 DEBUG:teuthology.parallel:result is None 2024-09-17T08:21:55.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:55 smithi203 ceph-mon[30259]: Updating smithi047:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:21:55.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:55 smithi203 ceph-mon[30259]: Updating smithi193:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:21:55.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:55 smithi203 ceph-mon[30259]: Updating smithi203:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:21:55.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:55 smithi203 ceph-mon[30259]: Deploying daemon alertmanager.a on smithi047 2024-09-17T08:21:55.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:55 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/1159278722' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-17T08:21:55.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:55 smithi047 ceph-mon[25866]: Updating smithi047:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:21:55.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:55 smithi047 ceph-mon[25866]: Updating smithi193:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:21:55.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:55 smithi047 ceph-mon[25866]: Updating smithi203:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:21:55.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:55 smithi047 ceph-mon[25866]: Deploying daemon alertmanager.a on smithi047 2024-09-17T08:21:55.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:55 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/1159278722' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-17T08:21:55.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:55 smithi193 ceph-mon[30002]: Updating smithi047:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:21:55.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:55 smithi193 ceph-mon[30002]: Updating smithi193:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:21:55.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:55 smithi193 ceph-mon[30002]: Updating smithi203:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:21:55.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:55 smithi193 ceph-mon[30002]: Deploying daemon alertmanager.a on smithi047 2024-09-17T08:21:55.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:55 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/1159278722' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-17T08:21:56.537 INFO:teuthology.orchestra.run.smithi047.stdout:77309411340 2024-09-17T08:21:56.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:56 smithi203 ceph-mon[30259]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-17T08:21:56.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:56 smithi047 ceph-mon[25866]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-17T08:21:56.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:56 smithi193 ceph-mon[30002]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-17T08:21:57.423 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411339 got 77309411340 for osd.2 2024-09-17T08:21:57.423 DEBUG:teuthology.parallel:result is None 2024-09-17T08:21:57.424 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-17T08:21:57.424 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:21:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:57.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:57 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/298325267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-17T08:21:57.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:57 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/298325267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-17T08:21:57.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:57 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/298325267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-17T08:21:58.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:21:58 smithi203 ceph-mon[30259]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T08:21:58.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:21:58 smithi047 ceph-mon[25866]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T08:21:58.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:21:58 smithi193 ceph-mon[30002]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T08:22:00.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:00 smithi203 ceph-mon[30259]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-17T08:22:00.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:00 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:22:00.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:00 smithi047 ceph-mon[25866]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-17T08:22:00.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:00 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:22:00.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:00 smithi193 ceph-mon[30002]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-17T08:22:00.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:00 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:22:00.984 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:22:00.986 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-09-17T08:22:01.624 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:01 smithi047 ceph-mon[25866]: from='client.14496 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:22:01.645 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-17T08:21:59.131510+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595838,"num_objects":193,"num_object_clones":0,"num_object_copies":579,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":193,"num_whiteouts":0,"num_read":962,"num_read_kb":809,"num_write":691,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":669,"ondisk_log_size":669,"up":387,"acting":387,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":387,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":281223168,"kb_used":100500,"kb_used_data":3060,"kb_used_omap":4,"kb_used_meta":97403,"kb_avail":281122668,"statfs":{"total":287972524032,"available":287869612032,"internally_reserved":0,"allocated":3133440,"data_stored":2099931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4775,"internal_metadata":99741017},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":206,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002977"},"pg_stats":[{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777186+0000","last_change":"2024-09-17T08:21:19.753809+0000","last_active":"2024-09-17T08:21:26.777186+0000","last_peered":"2024-09-17T08:21:26.777186+0000","last_clean":"2024-09-17T08:21:26.777186+0000","last_became_active":"2024-09-17T08:21:19.753364+0000","last_became_peered":"2024-09-17T08:21:19.753364+0000","last_unstale":"2024-09-17T08:21:26.777186+0000","last_undegraded":"2024-09-17T08:21:26.777186+0000","last_fullsized":"2024-09-17T08:21:26.777186+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:48:17.211239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'10","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.107482+0000","last_change":"2024-09-17T08:21:21.761298+0000","last_active":"2024-09-17T08:21:27.107482+0000","last_peered":"2024-09-17T08:21:27.107482+0000","last_clean":"2024-09-17T08:21:27.107482+0000","last_became_active":"2024-09-17T08:21:21.760143+0000","last_became_peered":"2024-09-17T08:21:21.760143+0000","last_unstale":"2024-09-17T08:21:27.107482+0000","last_undegraded":"2024-09-17T08:21:27.107482+0000","last_fullsized":"2024-09-17T08:21:27.107482+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:26:22.799002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776291+0000","last_change":"2024-09-17T08:21:23.766760+0000","last_active":"2024-09-17T08:21:26.776291+0000","last_peered":"2024-09-17T08:21:26.776291+0000","last_clean":"2024-09-17T08:21:26.776291+0000","last_became_active":"2024-09-17T08:21:23.766468+0000","last_became_peered":"2024-09-17T08:21:23.766468+0000","last_unstale":"2024-09-17T08:21:26.776291+0000","last_undegraded":"2024-09-17T08:21:26.776291+0000","last_fullsized":"2024-09-17T08:21:26.776291+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:15:56.495634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953709+0000","last_change":"2024-09-17T08:21:25.778378+0000","last_active":"2024-09-17T08:21:26.953709+0000","last_peered":"2024-09-17T08:21:26.953709+0000","last_clean":"2024-09-17T08:21:26.953709+0000","last_became_active":"2024-09-17T08:21:25.777985+0000","last_became_peered":"2024-09-17T08:21:25.777985+0000","last_unstale":"2024-09-17T08:21:26.953709+0000","last_undegraded":"2024-09-17T08:21:26.953709+0000","last_fullsized":"2024-09-17T08:21:26.953709+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:57.441771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777557+0000","last_change":"2024-09-17T08:21:19.754820+0000","last_active":"2024-09-17T08:21:26.777557+0000","last_peered":"2024-09-17T08:21:26.777557+0000","last_clean":"2024-09-17T08:21:26.777557+0000","last_became_active":"2024-09-17T08:21:19.753725+0000","last_became_peered":"2024-09-17T08:21:19.753725+0000","last_unstale":"2024-09-17T08:21:26.777557+0000","last_undegraded":"2024-09-17T08:21:26.777557+0000","last_fullsized":"2024-09-17T08:21:26.777557+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:59:38.573365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"30'11","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.114793+0000","last_change":"2024-09-17T08:21:21.764982+0000","last_active":"2024-09-17T08:21:27.114793+0000","last_peered":"2024-09-17T08:21:27.114793+0000","last_clean":"2024-09-17T08:21:27.114793+0000","last_became_active":"2024-09-17T08:21:21.764654+0000","last_became_peered":"2024-09-17T08:21:21.764654+0000","last_unstale":"2024-09-17T08:21:27.114793+0000","last_undegraded":"2024-09-17T08:21:27.114793+0000","last_fullsized":"2024-09-17T08:21:27.114793+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:03:06.826760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953747+0000","last_change":"2024-09-17T08:21:23.768701+0000","last_active":"2024-09-17T08:21:26.953747+0000","last_peered":"2024-09-17T08:21:26.953747+0000","last_clean":"2024-09-17T08:21:26.953747+0000","last_became_active":"2024-09-17T08:21:23.767948+0000","last_became_peered":"2024-09-17T08:21:23.767948+0000","last_unstale":"2024-09-17T08:21:26.953747+0000","last_undegraded":"2024-09-17T08:21:26.953747+0000","last_fullsized":"2024-09-17T08:21:26.953747+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:39.755523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:29.452509+0000","last_change":"2024-09-17T08:21:25.775451+0000","last_active":"2024-09-17T08:21:29.452509+0000","last_peered":"2024-09-17T08:21:29.452509+0000","last_clean":"2024-09-17T08:21:29.452509+0000","last_became_active":"2024-09-17T08:21:25.775174+0000","last_became_peered":"2024-09-17T08:21:25.775174+0000","last_unstale":"2024-09-17T08:21:29.452509+0000","last_undegraded":"2024-09-17T08:21:29.452509+0000","last_fullsized":"2024-09-17T08:21:29.452509+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:53.411212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777721+0000","last_change":"2024-09-17T08:21:19.754950+0000","last_active":"2024-09-17T08:21:26.777721+0000","last_peered":"2024-09-17T08:21:26.777721+0000","last_clean":"2024-09-17T08:21:26.777721+0000","last_became_active":"2024-09-17T08:21:19.754718+0000","last_became_peered":"2024-09-17T08:21:19.754718+0000","last_unstale":"2024-09-17T08:21:26.777721+0000","last_undegraded":"2024-09-17T08:21:26.777721+0000","last_fullsized":"2024-09-17T08:21:26.777721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:02:17.791289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"30'15","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.047608+0000","last_change":"2024-09-17T08:21:21.763471+0000","last_active":"2024-09-17T08:21:27.047608+0000","last_peered":"2024-09-17T08:21:27.047608+0000","last_clean":"2024-09-17T08:21:27.047608+0000","last_became_active":"2024-09-17T08:21:21.762999+0000","last_became_peered":"2024-09-17T08:21:21.762999+0000","last_unstale":"2024-09-17T08:21:27.047608+0000","last_undegraded":"2024-09-17T08:21:27.047608+0000","last_fullsized":"2024-09-17T08:21:27.047608+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:48.972836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953913+0000","last_change":"2024-09-17T08:21:23.768390+0000","last_active":"2024-09-17T08:21:26.953913+0000","last_peered":"2024-09-17T08:21:26.953913+0000","last_clean":"2024-09-17T08:21:26.953913+0000","last_became_active":"2024-09-17T08:21:23.767529+0000","last_became_peered":"2024-09-17T08:21:23.767529+0000","last_unstale":"2024-09-17T08:21:26.953913+0000","last_undegraded":"2024-09-17T08:21:26.953913+0000","last_fullsized":"2024-09-17T08:21:26.953913+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:17:44.539844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776409+0000","last_change":"2024-09-17T08:21:25.778355+0000","last_active":"2024-09-17T08:21:26.776409+0000","last_peered":"2024-09-17T08:21:26.776409+0000","last_clean":"2024-09-17T08:21:26.776409+0000","last_became_active":"2024-09-17T08:21:25.777977+0000","last_became_peered":"2024-09-17T08:21:25.777977+0000","last_unstale":"2024-09-17T08:21:26.776409+0000","last_undegraded":"2024-09-17T08:21:26.776409+0000","last_fullsized":"2024-09-17T08:21:26.776409+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:48:13.098229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777610+0000","last_change":"2024-09-17T08:21:19.753531+0000","last_active":"2024-09-17T08:21:26.777610+0000","last_peered":"2024-09-17T08:21:26.777610+0000","last_clean":"2024-09-17T08:21:26.777610+0000","last_became_active":"2024-09-17T08:21:19.753234+0000","last_became_peered":"2024-09-17T08:21:19.753234+0000","last_unstale":"2024-09-17T08:21:26.777610+0000","last_undegraded":"2024-09-17T08:21:26.777610+0000","last_fullsized":"2024-09-17T08:21:26.777610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:06.954989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"30'12","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.121506+0000","last_change":"2024-09-17T08:21:21.760540+0000","last_active":"2024-09-17T08:21:27.121506+0000","last_peered":"2024-09-17T08:21:27.121506+0000","last_clean":"2024-09-17T08:21:27.121506+0000","last_became_active":"2024-09-17T08:21:21.760259+0000","last_became_peered":"2024-09-17T08:21:21.760259+0000","last_unstale":"2024-09-17T08:21:27.121506+0000","last_undegraded":"2024-09-17T08:21:27.121506+0000","last_fullsized":"2024-09-17T08:21:27.121506+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:08:37.299230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953830+0000","last_change":"2024-09-17T08:21:23.767741+0000","last_active":"2024-09-17T08:21:26.953830+0000","last_peered":"2024-09-17T08:21:26.953830+0000","last_clean":"2024-09-17T08:21:26.953830+0000","last_became_active":"2024-09-17T08:21:23.767269+0000","last_became_peered":"2024-09-17T08:21:23.767269+0000","last_unstale":"2024-09-17T08:21:26.953830+0000","last_undegraded":"2024-09-17T08:21:26.953830+0000","last_fullsized":"2024-09-17T08:21:26.953830+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:25:14.349907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953868+0000","last_change":"2024-09-17T08:21:25.777936+0000","last_active":"2024-09-17T08:21:26.953868+0000","last_peered":"2024-09-17T08:21:26.953868+0000","last_clean":"2024-09-17T08:21:26.953868+0000","last_became_active":"2024-09-17T08:21:25.777140+0000","last_became_peered":"2024-09-17T08:21:25.777140+0000","last_unstale":"2024-09-17T08:21:26.953868+0000","last_undegraded":"2024-09-17T08:21:26.953868+0000","last_fullsized":"2024-09-17T08:21:26.953868+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:26:55.860102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"30'19","reported_seq":47,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.125658+0000","last_change":"2024-09-17T08:21:21.763336+0000","last_active":"2024-09-17T08:21:27.125658+0000","last_peered":"2024-09-17T08:21:27.125658+0000","last_clean":"2024-09-17T08:21:27.125658+0000","last_became_active":"2024-09-17T08:21:21.762352+0000","last_became_peered":"2024-09-17T08:21:21.762352+0000","last_unstale":"2024-09-17T08:21:27.125658+0000","last_undegraded":"2024-09-17T08:21:27.125658+0000","last_fullsized":"2024-09-17T08:21:27.125658+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:01.333731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777036+0000","last_change":"2024-09-17T08:21:19.753817+0000","last_active":"2024-09-17T08:21:26.777036+0000","last_peered":"2024-09-17T08:21:26.777036+0000","last_clean":"2024-09-17T08:21:26.777036+0000","last_became_active":"2024-09-17T08:21:19.753539+0000","last_became_peered":"2024-09-17T08:21:19.753539+0000","last_unstale":"2024-09-17T08:21:26.777036+0000","last_undegraded":"2024-09-17T08:21:26.777036+0000","last_fullsized":"2024-09-17T08:21:26.777036+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:49:29.345564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"30'8","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T08:21:47.899461+0000","last_change":"2024-09-17T08:21:23.768193+0000","last_active":"2024-09-17T08:21:47.899461+0000","last_peered":"2024-09-17T08:21:47.899461+0000","last_clean":"2024-09-17T08:21:47.899461+0000","last_became_active":"2024-09-17T08:21:23.767403+0000","last_became_peered":"2024-09-17T08:21:23.767403+0000","last_unstale":"2024-09-17T08:21:47.899461+0000","last_undegraded":"2024-09-17T08:21:47.899461+0000","last_fullsized":"2024-09-17T08:21:47.899461+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:21:00.514569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:29.452704+0000","last_change":"2024-09-17T08:21:25.781641+0000","last_active":"2024-09-17T08:21:29.452704+0000","last_peered":"2024-09-17T08:21:29.452704+0000","last_clean":"2024-09-17T08:21:29.452704+0000","last_became_active":"2024-09-17T08:21:25.781062+0000","last_became_peered":"2024-09-17T08:21:25.781062+0000","last_unstale":"2024-09-17T08:21:29.452704+0000","last_undegraded":"2024-09-17T08:21:29.452704+0000","last_fullsized":"2024-09-17T08:21:29.452704+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:04.115389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.082760+0000","last_change":"2024-09-17T08:21:21.761087+0000","last_active":"2024-09-17T08:21:27.082760+0000","last_peered":"2024-09-17T08:21:27.082760+0000","last_clean":"2024-09-17T08:21:27.082760+0000","last_became_active":"2024-09-17T08:21:21.760742+0000","last_became_peered":"2024-09-17T08:21:21.760742+0000","last_unstale":"2024-09-17T08:21:27.082760+0000","last_undegraded":"2024-09-17T08:21:27.082760+0000","last_fullsized":"2024-09-17T08:21:27.082760+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:14.224261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'1","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.876282+0000","last_change":"2024-09-17T08:21:19.753880+0000","last_active":"2024-09-17T08:21:26.876282+0000","last_peered":"2024-09-17T08:21:26.876282+0000","last_clean":"2024-09-17T08:21:26.876282+0000","last_became_active":"2024-09-17T08:21:19.753604+0000","last_became_peered":"2024-09-17T08:21:19.753604+0000","last_unstale":"2024-09-17T08:21:26.876282+0000","last_undegraded":"2024-09-17T08:21:26.876282+0000","last_fullsized":"2024-09-17T08:21:26.876282+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:39:29.877304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953948+0000","last_change":"2024-09-17T08:21:23.768481+0000","last_active":"2024-09-17T08:21:26.953948+0000","last_peered":"2024-09-17T08:21:26.953948+0000","last_clean":"2024-09-17T08:21:26.953948+0000","last_became_active":"2024-09-17T08:21:23.767793+0000","last_became_peered":"2024-09-17T08:21:23.767793+0000","last_unstale":"2024-09-17T08:21:26.953948+0000","last_undegraded":"2024-09-17T08:21:26.953948+0000","last_fullsized":"2024-09-17T08:21:26.953948+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:30.754360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.778677+0000","last_change":"2024-09-17T08:21:25.779077+0000","last_active":"2024-09-17T08:21:26.778677+0000","last_peered":"2024-09-17T08:21:26.778677+0000","last_clean":"2024-09-17T08:21:26.778677+0000","last_became_active":"2024-09-17T08:21:25.778766+0000","last_became_peered":"2024-09-17T08:21:25.778766+0000","last_unstale":"2024-09-17T08:21:26.778677+0000","last_undegraded":"2024-09-17T08:21:26.778677+0000","last_fullsized":"2024-09-17T08:21:26.778677+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:22.641911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"30'15","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.054262+0000","last_change":"2024-09-17T08:21:21.758855+0000","last_active":"2024-09-17T08:21:27.054262+0000","last_peered":"2024-09-17T08:21:27.054262+0000","last_clean":"2024-09-17T08:21:27.054262+0000","last_became_active":"2024-09-17T08:21:21.758541+0000","last_became_peered":"2024-09-17T08:21:21.758541+0000","last_unstale":"2024-09-17T08:21:27.054262+0000","last_undegraded":"2024-09-17T08:21:27.054262+0000","last_fullsized":"2024-09-17T08:21:27.054262+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:20.421507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.775423+0000","last_change":"2024-09-17T08:21:19.753409+0000","last_active":"2024-09-17T08:21:26.775423+0000","last_peered":"2024-09-17T08:21:26.775423+0000","last_clean":"2024-09-17T08:21:26.775423+0000","last_became_active":"2024-09-17T08:21:19.752680+0000","last_became_peered":"2024-09-17T08:21:19.752680+0000","last_unstale":"2024-09-17T08:21:26.775423+0000","last_undegraded":"2024-09-17T08:21:26.775423+0000","last_fullsized":"2024-09-17T08:21:26.775423+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:25.249461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777625+0000","last_change":"2024-09-17T08:21:23.768240+0000","last_active":"2024-09-17T08:21:26.777625+0000","last_peered":"2024-09-17T08:21:26.777625+0000","last_clean":"2024-09-17T08:21:26.777625+0000","last_became_active":"2024-09-17T08:21:23.767892+0000","last_became_peered":"2024-09-17T08:21:23.767892+0000","last_unstale":"2024-09-17T08:21:26.777625+0000","last_undegraded":"2024-09-17T08:21:26.777625+0000","last_fullsized":"2024-09-17T08:21:26.777625+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:00.201079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.778555+0000","last_change":"2024-09-17T08:21:25.779085+0000","last_active":"2024-09-17T08:21:26.778555+0000","last_peered":"2024-09-17T08:21:26.778555+0000","last_clean":"2024-09-17T08:21:26.778555+0000","last_became_active":"2024-09-17T08:21:25.778814+0000","last_became_peered":"2024-09-17T08:21:25.778814+0000","last_unstale":"2024-09-17T08:21:26.778555+0000","last_undegraded":"2024-09-17T08:21:26.778555+0000","last_fullsized":"2024-09-17T08:21:26.778555+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:30:54.657744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"30'12","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.927669+0000","last_change":"2024-09-17T08:21:21.760367+0000","last_active":"2024-09-17T08:21:26.927669+0000","last_peered":"2024-09-17T08:21:26.927669+0000","last_clean":"2024-09-17T08:21:26.927669+0000","last_became_active":"2024-09-17T08:21:21.759924+0000","last_became_peered":"2024-09-17T08:21:21.759924+0000","last_unstale":"2024-09-17T08:21:26.927669+0000","last_undegraded":"2024-09-17T08:21:26.927669+0000","last_fullsized":"2024-09-17T08:21:26.927669+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:50:31.120594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.779058+0000","last_change":"2024-09-17T08:21:19.755788+0000","last_active":"2024-09-17T08:21:26.779058+0000","last_peered":"2024-09-17T08:21:26.779058+0000","last_clean":"2024-09-17T08:21:26.779058+0000","last_became_active":"2024-09-17T08:21:19.755266+0000","last_became_peered":"2024-09-17T08:21:19.755266+0000","last_unstale":"2024-09-17T08:21:26.779058+0000","last_undegraded":"2024-09-17T08:21:26.779058+0000","last_fullsized":"2024-09-17T08:21:26.779058+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:09.532510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"30'8","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T08:21:47.899455+0000","last_change":"2024-09-17T08:21:23.770164+0000","last_active":"2024-09-17T08:21:47.899455+0000","last_peered":"2024-09-17T08:21:47.899455+0000","last_clean":"2024-09-17T08:21:47.899455+0000","last_became_active":"2024-09-17T08:21:23.769849+0000","last_became_peered":"2024-09-17T08:21:23.769849+0000","last_unstale":"2024-09-17T08:21:47.899455+0000","last_undegraded":"2024-09-17T08:21:47.899455+0000","last_fullsized":"2024-09-17T08:21:47.899455+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:40:11.343064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.954052+0000","last_change":"2024-09-17T08:21:25.779176+0000","last_active":"2024-09-17T08:21:26.954052+0000","last_peered":"2024-09-17T08:21:26.954052+0000","last_clean":"2024-09-17T08:21:26.954052+0000","last_became_active":"2024-09-17T08:21:25.778843+0000","last_became_peered":"2024-09-17T08:21:25.778843+0000","last_unstale":"2024-09-17T08:21:26.954052+0000","last_undegraded":"2024-09-17T08:21:26.954052+0000","last_fullsized":"2024-09-17T08:21:26.954052+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:32:51.078525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"30'12","reported_seq":34,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.045349+0000","last_change":"2024-09-17T08:21:21.760527+0000","last_active":"2024-09-17T08:21:27.045349+0000","last_peered":"2024-09-17T08:21:27.045349+0000","last_clean":"2024-09-17T08:21:27.045349+0000","last_became_active":"2024-09-17T08:21:21.760176+0000","last_became_peered":"2024-09-17T08:21:21.760176+0000","last_unstale":"2024-09-17T08:21:27.045349+0000","last_undegraded":"2024-09-17T08:21:27.045349+0000","last_fullsized":"2024-09-17T08:21:27.045349+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:15.048390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.780703+0000","last_change":"2024-09-17T08:21:19.753685+0000","last_active":"2024-09-17T08:21:26.780703+0000","last_peered":"2024-09-17T08:21:26.780703+0000","last_clean":"2024-09-17T08:21:26.780703+0000","last_became_active":"2024-09-17T08:21:19.753316+0000","last_became_peered":"2024-09-17T08:21:19.753316+0000","last_unstale":"2024-09-17T08:21:26.780703+0000","last_undegraded":"2024-09-17T08:21:26.780703+0000","last_fullsized":"2024-09-17T08:21:26.780703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:43.053033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776918+0000","last_change":"2024-09-17T08:21:23.764057+0000","last_active":"2024-09-17T08:21:26.776918+0000","last_peered":"2024-09-17T08:21:26.776918+0000","last_clean":"2024-09-17T08:21:26.776918+0000","last_became_active":"2024-09-17T08:21:23.763794+0000","last_became_peered":"2024-09-17T08:21:23.763794+0000","last_unstale":"2024-09-17T08:21:26.776918+0000","last_undegraded":"2024-09-17T08:21:26.776918+0000","last_fullsized":"2024-09-17T08:21:26.776918+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:18:15.786269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776013+0000","last_change":"2024-09-17T08:21:25.779140+0000","last_active":"2024-09-17T08:21:26.776013+0000","last_peered":"2024-09-17T08:21:26.776013+0000","last_clean":"2024-09-17T08:21:26.776013+0000","last_became_active":"2024-09-17T08:21:25.778645+0000","last_became_peered":"2024-09-17T08:21:25.778645+0000","last_unstale":"2024-09-17T08:21:26.776013+0000","last_undegraded":"2024-09-17T08:21:26.776013+0000","last_fullsized":"2024-09-17T08:21:26.776013+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:06.498952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"30'13","reported_seq":43,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.035148+0000","last_change":"2024-09-17T08:21:21.765897+0000","last_active":"2024-09-17T08:21:27.035148+0000","last_peered":"2024-09-17T08:21:27.035148+0000","last_clean":"2024-09-17T08:21:27.035148+0000","last_became_active":"2024-09-17T08:21:21.765433+0000","last_became_peered":"2024-09-17T08:21:21.765433+0000","last_unstale":"2024-09-17T08:21:27.035148+0000","last_undegraded":"2024-09-17T08:21:27.035148+0000","last_fullsized":"2024-09-17T08:21:27.035148+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:16:23.526808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777650+0000","last_change":"2024-09-17T08:21:19.754523+0000","last_active":"2024-09-17T08:21:26.777650+0000","last_peered":"2024-09-17T08:21:26.777650+0000","last_clean":"2024-09-17T08:21:26.777650+0000","last_became_active":"2024-09-17T08:21:19.754065+0000","last_became_peered":"2024-09-17T08:21:19.754065+0000","last_unstale":"2024-09-17T08:21:26.777650+0000","last_undegraded":"2024-09-17T08:21:26.777650+0000","last_fullsized":"2024-09-17T08:21:26.777650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:02.566708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"30'8","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T08:21:47.899317+0000","last_change":"2024-09-17T08:21:23.768550+0000","last_active":"2024-09-17T08:21:47.899317+0000","last_peered":"2024-09-17T08:21:47.899317+0000","last_clean":"2024-09-17T08:21:47.899317+0000","last_became_active":"2024-09-17T08:21:23.768234+0000","last_became_peered":"2024-09-17T08:21:23.768234+0000","last_unstale":"2024-09-17T08:21:47.899317+0000","last_undegraded":"2024-09-17T08:21:47.899317+0000","last_fullsized":"2024-09-17T08:21:47.899317+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:16.315747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953797+0000","last_change":"2024-09-17T08:21:25.779665+0000","last_active":"2024-09-17T08:21:26.953797+0000","last_peered":"2024-09-17T08:21:26.953797+0000","last_clean":"2024-09-17T08:21:26.953797+0000","last_became_active":"2024-09-17T08:21:25.779395+0000","last_became_peered":"2024-09-17T08:21:25.779395+0000","last_unstale":"2024-09-17T08:21:26.953797+0000","last_undegraded":"2024-09-17T08:21:26.953797+0000","last_fullsized":"2024-09-17T08:21:26.953797+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:02:17.500347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"30'28","reported_seq":71,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T08:21:47.899382+0000","last_change":"2024-09-17T08:21:21.761259+0000","last_active":"2024-09-17T08:21:47.899382+0000","last_peered":"2024-09-17T08:21:47.899382+0000","last_clean":"2024-09-17T08:21:47.899382+0000","last_became_active":"2024-09-17T08:21:21.760837+0000","last_became_peered":"2024-09-17T08:21:21.760837+0000","last_unstale":"2024-09-17T08:21:47.899382+0000","last_undegraded":"2024-09-17T08:21:47.899382+0000","last_fullsized":"2024-09-17T08:21:47.899382+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:09:23.512446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777594+0000","last_change":"2024-09-17T08:21:19.755691+0000","last_active":"2024-09-17T08:21:26.777594+0000","last_peered":"2024-09-17T08:21:26.777594+0000","last_clean":"2024-09-17T08:21:26.777594+0000","last_became_active":"2024-09-17T08:21:19.755459+0000","last_became_peered":"2024-09-17T08:21:19.755459+0000","last_unstale":"2024-09-17T08:21:26.777594+0000","last_undegraded":"2024-09-17T08:21:26.777594+0000","last_fullsized":"2024-09-17T08:21:26.777594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:53:59.366268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"30'8","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T08:21:47.899711+0000","last_change":"2024-09-17T08:21:23.767521+0000","last_active":"2024-09-17T08:21:47.899711+0000","last_peered":"2024-09-17T08:21:47.899711+0000","last_clean":"2024-09-17T08:21:47.899711+0000","last_became_active":"2024-09-17T08:21:23.767240+0000","last_became_peered":"2024-09-17T08:21:23.767240+0000","last_unstale":"2024-09-17T08:21:47.899711+0000","last_undegraded":"2024-09-17T08:21:47.899711+0000","last_fullsized":"2024-09-17T08:21:47.899711+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:55:50.735309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953850+0000","last_change":"2024-09-17T08:21:25.778372+0000","last_active":"2024-09-17T08:21:26.953850+0000","last_peered":"2024-09-17T08:21:26.953850+0000","last_clean":"2024-09-17T08:21:26.953850+0000","last_became_active":"2024-09-17T08:21:25.778033+0000","last_became_peered":"2024-09-17T08:21:25.778033+0000","last_unstale":"2024-09-17T08:21:26.953850+0000","last_undegraded":"2024-09-17T08:21:26.953850+0000","last_fullsized":"2024-09-17T08:21:26.953850+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:06:03.503230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"30'14","reported_seq":38,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.100761+0000","last_change":"2024-09-17T08:21:21.763296+0000","last_active":"2024-09-17T08:21:27.100761+0000","last_peered":"2024-09-17T08:21:27.100761+0000","last_clean":"2024-09-17T08:21:27.100761+0000","last_became_active":"2024-09-17T08:21:21.762228+0000","last_became_peered":"2024-09-17T08:21:21.762228+0000","last_unstale":"2024-09-17T08:21:27.100761+0000","last_undegraded":"2024-09-17T08:21:27.100761+0000","last_fullsized":"2024-09-17T08:21:27.100761+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:10.542907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776591+0000","last_change":"2024-09-17T08:21:19.753733+0000","last_active":"2024-09-17T08:21:26.776591+0000","last_peered":"2024-09-17T08:21:26.776591+0000","last_clean":"2024-09-17T08:21:26.776591+0000","last_became_active":"2024-09-17T08:21:19.753192+0000","last_became_peered":"2024-09-17T08:21:19.753192+0000","last_unstale":"2024-09-17T08:21:26.776591+0000","last_undegraded":"2024-09-17T08:21:26.776591+0000","last_fullsized":"2024-09-17T08:21:26.776591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:50.090763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777503+0000","last_change":"2024-09-17T08:21:23.766922+0000","last_active":"2024-09-17T08:21:26.777503+0000","last_peered":"2024-09-17T08:21:26.777503+0000","last_clean":"2024-09-17T08:21:26.777503+0000","last_became_active":"2024-09-17T08:21:23.766628+0000","last_became_peered":"2024-09-17T08:21:23.766628+0000","last_unstale":"2024-09-17T08:21:26.777503+0000","last_undegraded":"2024-09-17T08:21:26.777503+0000","last_fullsized":"2024-09-17T08:21:26.777503+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:56:03.480056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:29.452622+0000","last_change":"2024-09-17T08:21:25.780233+0000","last_active":"2024-09-17T08:21:29.452622+0000","last_peered":"2024-09-17T08:21:29.452622+0000","last_clean":"2024-09-17T08:21:29.452622+0000","last_became_active":"2024-09-17T08:21:25.779528+0000","last_became_peered":"2024-09-17T08:21:25.779528+0000","last_unstale":"2024-09-17T08:21:29.452622+0000","last_undegraded":"2024-09-17T08:21:29.452622+0000","last_fullsized":"2024-09-17T08:21:29.452622+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:41:26.599909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"30'19","reported_seq":52,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.087683+0000","last_change":"2024-09-17T08:21:21.760953+0000","last_active":"2024-09-17T08:21:27.087683+0000","last_peered":"2024-09-17T08:21:27.087683+0000","last_clean":"2024-09-17T08:21:27.087683+0000","last_became_active":"2024-09-17T08:21:21.760470+0000","last_became_peered":"2024-09-17T08:21:21.760470+0000","last_unstale":"2024-09-17T08:21:27.087683+0000","last_undegraded":"2024-09-17T08:21:27.087683+0000","last_fullsized":"2024-09-17T08:21:27.087683+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:06.666050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.779017+0000","last_change":"2024-09-17T08:21:19.753512+0000","last_active":"2024-09-17T08:21:26.779017+0000","last_peered":"2024-09-17T08:21:26.779017+0000","last_clean":"2024-09-17T08:21:26.779017+0000","last_became_active":"2024-09-17T08:21:19.752865+0000","last_became_peered":"2024-09-17T08:21:19.752865+0000","last_unstale":"2024-09-17T08:21:26.779017+0000","last_undegraded":"2024-09-17T08:21:26.779017+0000","last_fullsized":"2024-09-17T08:21:26.779017+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:50.469031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777088+0000","last_change":"2024-09-17T08:21:23.768280+0000","last_active":"2024-09-17T08:21:26.777088+0000","last_peered":"2024-09-17T08:21:26.777088+0000","last_clean":"2024-09-17T08:21:26.777088+0000","last_became_active":"2024-09-17T08:21:23.767885+0000","last_became_peered":"2024-09-17T08:21:23.767885+0000","last_unstale":"2024-09-17T08:21:26.777088+0000","last_undegraded":"2024-09-17T08:21:26.777088+0000","last_fullsized":"2024-09-17T08:21:26.777088+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:49.716822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:29.452776+0000","last_change":"2024-09-17T08:21:25.781133+0000","last_active":"2024-09-17T08:21:29.452776+0000","last_peered":"2024-09-17T08:21:29.452776+0000","last_clean":"2024-09-17T08:21:29.452776+0000","last_became_active":"2024-09-17T08:21:25.780757+0000","last_became_peered":"2024-09-17T08:21:25.780757+0000","last_unstale":"2024-09-17T08:21:29.452776+0000","last_undegraded":"2024-09-17T08:21:29.452776+0000","last_fullsized":"2024-09-17T08:21:29.452776+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:31.081090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"30'18","reported_seq":48,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.076522+0000","last_change":"2024-09-17T08:21:21.760535+0000","last_active":"2024-09-17T08:21:27.076522+0000","last_peered":"2024-09-17T08:21:27.076522+0000","last_clean":"2024-09-17T08:21:27.076522+0000","last_became_active":"2024-09-17T08:21:21.760237+0000","last_became_peered":"2024-09-17T08:21:21.760237+0000","last_unstale":"2024-09-17T08:21:27.076522+0000","last_undegraded":"2024-09-17T08:21:27.076522+0000","last_fullsized":"2024-09-17T08:21:27.076522+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:34:19.786977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.775716+0000","last_change":"2024-09-17T08:21:19.754333+0000","last_active":"2024-09-17T08:21:26.775716+0000","last_peered":"2024-09-17T08:21:26.775716+0000","last_clean":"2024-09-17T08:21:26.775716+0000","last_became_active":"2024-09-17T08:21:19.753294+0000","last_became_peered":"2024-09-17T08:21:19.753294+0000","last_unstale":"2024-09-17T08:21:26.775716+0000","last_undegraded":"2024-09-17T08:21:26.775716+0000","last_fullsized":"2024-09-17T08:21:26.775716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:49:01.893917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777288+0000","last_change":"2024-09-17T08:21:23.767277+0000","last_active":"2024-09-17T08:21:26.777288+0000","last_peered":"2024-09-17T08:21:26.777288+0000","last_clean":"2024-09-17T08:21:26.777288+0000","last_became_active":"2024-09-17T08:21:23.766902+0000","last_became_peered":"2024-09-17T08:21:23.766902+0000","last_unstale":"2024-09-17T08:21:26.777288+0000","last_undegraded":"2024-09-17T08:21:26.777288+0000","last_fullsized":"2024-09-17T08:21:26.777288+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:44:51.770796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:29.452826+0000","last_change":"2024-09-17T08:21:25.780230+0000","last_active":"2024-09-17T08:21:29.452826+0000","last_peered":"2024-09-17T08:21:29.452826+0000","last_clean":"2024-09-17T08:21:29.452826+0000","last_became_active":"2024-09-17T08:21:25.779909+0000","last_became_peered":"2024-09-17T08:21:25.779909+0000","last_unstale":"2024-09-17T08:21:29.452826+0000","last_undegraded":"2024-09-17T08:21:29.452826+0000","last_fullsized":"2024-09-17T08:21:29.452826+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:09:26.699991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"30'14","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.109828+0000","last_change":"2024-09-17T08:21:21.761155+0000","last_active":"2024-09-17T08:21:27.109828+0000","last_peered":"2024-09-17T08:21:27.109828+0000","last_clean":"2024-09-17T08:21:27.109828+0000","last_became_active":"2024-09-17T08:21:21.760070+0000","last_became_peered":"2024-09-17T08:21:21.760070+0000","last_unstale":"2024-09-17T08:21:27.109828+0000","last_undegraded":"2024-09-17T08:21:27.109828+0000","last_fullsized":"2024-09-17T08:21:27.109828+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:25.057234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.778960+0000","last_change":"2024-09-17T08:21:19.753672+0000","last_active":"2024-09-17T08:21:26.778960+0000","last_peered":"2024-09-17T08:21:26.778960+0000","last_clean":"2024-09-17T08:21:26.778960+0000","last_became_active":"2024-09-17T08:21:19.753187+0000","last_became_peered":"2024-09-17T08:21:19.753187+0000","last_unstale":"2024-09-17T08:21:26.778960+0000","last_undegraded":"2024-09-17T08:21:26.778960+0000","last_fullsized":"2024-09-17T08:21:26.778960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:55.086952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777481+0000","last_change":"2024-09-17T08:21:23.764097+0000","last_active":"2024-09-17T08:21:26.777481+0000","last_peered":"2024-09-17T08:21:26.777481+0000","last_clean":"2024-09-17T08:21:26.777481+0000","last_became_active":"2024-09-17T08:21:23.763769+0000","last_became_peered":"2024-09-17T08:21:23.763769+0000","last_unstale":"2024-09-17T08:21:26.777481+0000","last_undegraded":"2024-09-17T08:21:26.777481+0000","last_fullsized":"2024-09-17T08:21:26.777481+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:22:28.012538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.954083+0000","last_change":"2024-09-17T08:21:25.777566+0000","last_active":"2024-09-17T08:21:26.954083+0000","last_peered":"2024-09-17T08:21:26.954083+0000","last_clean":"2024-09-17T08:21:26.954083+0000","last_became_active":"2024-09-17T08:21:25.777255+0000","last_became_peered":"2024-09-17T08:21:25.777255+0000","last_unstale":"2024-09-17T08:21:26.954083+0000","last_undegraded":"2024-09-17T08:21:26.954083+0000","last_fullsized":"2024-09-17T08:21:26.954083+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:26:37.643645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"30'10","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.095275+0000","last_change":"2024-09-17T08:21:21.760931+0000","last_active":"2024-09-17T08:21:27.095275+0000","last_peered":"2024-09-17T08:21:27.095275+0000","last_clean":"2024-09-17T08:21:27.095275+0000","last_became_active":"2024-09-17T08:21:21.760489+0000","last_became_peered":"2024-09-17T08:21:21.760489+0000","last_unstale":"2024-09-17T08:21:27.095275+0000","last_undegraded":"2024-09-17T08:21:27.095275+0000","last_fullsized":"2024-09-17T08:21:27.095275+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:34.786006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.779553+0000","last_change":"2024-09-17T08:21:19.753402+0000","last_active":"2024-09-17T08:21:26.779553+0000","last_peered":"2024-09-17T08:21:26.779553+0000","last_clean":"2024-09-17T08:21:26.779553+0000","last_became_active":"2024-09-17T08:21:19.753000+0000","last_became_peered":"2024-09-17T08:21:19.753000+0000","last_unstale":"2024-09-17T08:21:26.779553+0000","last_undegraded":"2024-09-17T08:21:26.779553+0000","last_fullsized":"2024-09-17T08:21:26.779553+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:55:13.233754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"31'208","reported_seq":302,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T08:21:47.194383+0000","last_change":"2024-09-17T08:21:07.119870+0000","last_active":"2024-09-17T08:21:47.194383+0000","last_peered":"2024-09-17T08:21:47.194383+0000","last_clean":"2024-09-17T08:21:47.194383+0000","last_became_active":"2024-09-17T08:21:07.118876+0000","last_became_peered":"2024-09-17T08:21:07.118876+0000","last_unstale":"2024-09-17T08:21:47.194383+0000","last_undegraded":"2024-09-17T08:21:47.194383+0000","last_fullsized":"2024-09-17T08:21:47.194383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:06.110207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:06.110207+0000","last_clean_scrub_stamp":"2024-09-17T08:21:06.110207+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:21.814379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953989+0000","last_change":"2024-09-17T08:21:23.768358+0000","last_active":"2024-09-17T08:21:26.953989+0000","last_peered":"2024-09-17T08:21:26.953989+0000","last_clean":"2024-09-17T08:21:26.953989+0000","last_became_active":"2024-09-17T08:21:23.767553+0000","last_became_peered":"2024-09-17T08:21:23.767553+0000","last_unstale":"2024-09-17T08:21:26.953989+0000","last_undegraded":"2024-09-17T08:21:26.953989+0000","last_fullsized":"2024-09-17T08:21:26.953989+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:34.815653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777755+0000","last_change":"2024-09-17T08:21:25.779201+0000","last_active":"2024-09-17T08:21:26.777755+0000","last_peered":"2024-09-17T08:21:26.777755+0000","last_clean":"2024-09-17T08:21:26.777755+0000","last_became_active":"2024-09-17T08:21:25.778435+0000","last_became_peered":"2024-09-17T08:21:25.778435+0000","last_unstale":"2024-09-17T08:21:26.777755+0000","last_undegraded":"2024-09-17T08:21:26.777755+0000","last_fullsized":"2024-09-17T08:21:26.777755+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:55:19.814835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'17","reported_seq":44,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.102894+0000","last_change":"2024-09-17T08:21:21.760682+0000","last_active":"2024-09-17T08:21:27.102894+0000","last_peered":"2024-09-17T08:21:27.102894+0000","last_clean":"2024-09-17T08:21:27.102894+0000","last_became_active":"2024-09-17T08:21:21.760171+0000","last_became_peered":"2024-09-17T08:21:21.760171+0000","last_unstale":"2024-09-17T08:21:27.102894+0000","last_undegraded":"2024-09-17T08:21:27.102894+0000","last_fullsized":"2024-09-17T08:21:27.102894+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:44.217636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776861+0000","last_change":"2024-09-17T08:21:19.753083+0000","last_active":"2024-09-17T08:21:26.776861+0000","last_peered":"2024-09-17T08:21:26.776861+0000","last_clean":"2024-09-17T08:21:26.776861+0000","last_became_active":"2024-09-17T08:21:19.752793+0000","last_became_peered":"2024-09-17T08:21:19.752793+0000","last_unstale":"2024-09-17T08:21:26.776861+0000","last_undegraded":"2024-09-17T08:21:26.776861+0000","last_fullsized":"2024-09-17T08:21:26.776861+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:41:31.625936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953840+0000","last_change":"2024-09-17T08:21:23.768588+0000","last_active":"2024-09-17T08:21:26.953840+0000","last_peered":"2024-09-17T08:21:26.953840+0000","last_clean":"2024-09-17T08:21:26.953840+0000","last_became_active":"2024-09-17T08:21:23.768078+0000","last_became_peered":"2024-09-17T08:21:23.768078+0000","last_unstale":"2024-09-17T08:21:26.953840+0000","last_undegraded":"2024-09-17T08:21:26.953840+0000","last_fullsized":"2024-09-17T08:21:26.953840+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:38.348121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777549+0000","last_change":"2024-09-17T08:21:25.779025+0000","last_active":"2024-09-17T08:21:26.777549+0000","last_peered":"2024-09-17T08:21:26.777549+0000","last_clean":"2024-09-17T08:21:26.777549+0000","last_became_active":"2024-09-17T08:21:25.778664+0000","last_became_peered":"2024-09-17T08:21:25.778664+0000","last_unstale":"2024-09-17T08:21:26.777549+0000","last_undegraded":"2024-09-17T08:21:26.777549+0000","last_fullsized":"2024-09-17T08:21:26.777549+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:31:57.823770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"30'10","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.056688+0000","last_change":"2024-09-17T08:21:21.760419+0000","last_active":"2024-09-17T08:21:27.056688+0000","last_peered":"2024-09-17T08:21:27.056688+0000","last_clean":"2024-09-17T08:21:27.056688+0000","last_became_active":"2024-09-17T08:21:21.760102+0000","last_became_peered":"2024-09-17T08:21:21.760102+0000","last_unstale":"2024-09-17T08:21:27.056688+0000","last_undegraded":"2024-09-17T08:21:27.056688+0000","last_fullsized":"2024-09-17T08:21:27.056688+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:49.052441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777303+0000","last_change":"2024-09-17T08:21:19.755785+0000","last_active":"2024-09-17T08:21:26.777303+0000","last_peered":"2024-09-17T08:21:26.777303+0000","last_clean":"2024-09-17T08:21:26.777303+0000","last_became_active":"2024-09-17T08:21:19.755490+0000","last_became_peered":"2024-09-17T08:21:19.755490+0000","last_unstale":"2024-09-17T08:21:26.777303+0000","last_undegraded":"2024-09-17T08:21:26.777303+0000","last_fullsized":"2024-09-17T08:21:26.777303+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:59.056786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776568+0000","last_change":"2024-09-17T08:21:23.767056+0000","last_active":"2024-09-17T08:21:26.776568+0000","last_peered":"2024-09-17T08:21:26.776568+0000","last_clean":"2024-09-17T08:21:26.776568+0000","last_became_active":"2024-09-17T08:21:23.766760+0000","last_became_peered":"2024-09-17T08:21:23.766760+0000","last_unstale":"2024-09-17T08:21:26.776568+0000","last_undegraded":"2024-09-17T08:21:26.776568+0000","last_fullsized":"2024-09-17T08:21:26.776568+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:43.136151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:29.452565+0000","last_change":"2024-09-17T08:21:25.774852+0000","last_active":"2024-09-17T08:21:29.452565+0000","last_peered":"2024-09-17T08:21:29.452565+0000","last_clean":"2024-09-17T08:21:29.452565+0000","last_became_active":"2024-09-17T08:21:25.774558+0000","last_became_peered":"2024-09-17T08:21:25.774558+0000","last_unstale":"2024-09-17T08:21:29.452565+0000","last_undegraded":"2024-09-17T08:21:29.452565+0000","last_fullsized":"2024-09-17T08:21:29.452565+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:36.454496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"30'15","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.127501+0000","last_change":"2024-09-17T08:21:21.763292+0000","last_active":"2024-09-17T08:21:27.127501+0000","last_peered":"2024-09-17T08:21:27.127501+0000","last_clean":"2024-09-17T08:21:27.127501+0000","last_became_active":"2024-09-17T08:21:21.762260+0000","last_became_peered":"2024-09-17T08:21:21.762260+0000","last_unstale":"2024-09-17T08:21:27.127501+0000","last_undegraded":"2024-09-17T08:21:27.127501+0000","last_fullsized":"2024-09-17T08:21:27.127501+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:14.413275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777464+0000","last_change":"2024-09-17T08:21:19.754334+0000","last_active":"2024-09-17T08:21:26.777464+0000","last_peered":"2024-09-17T08:21:26.777464+0000","last_clean":"2024-09-17T08:21:26.777464+0000","last_became_active":"2024-09-17T08:21:19.753029+0000","last_became_peered":"2024-09-17T08:21:19.753029+0000","last_unstale":"2024-09-17T08:21:26.777464+0000","last_undegraded":"2024-09-17T08:21:26.777464+0000","last_fullsized":"2024-09-17T08:21:26.777464+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:16:19.205841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777100+0000","last_change":"2024-09-17T08:21:23.768393+0000","last_active":"2024-09-17T08:21:26.777100+0000","last_peered":"2024-09-17T08:21:26.777100+0000","last_clean":"2024-09-17T08:21:26.777100+0000","last_became_active":"2024-09-17T08:21:23.767769+0000","last_became_peered":"2024-09-17T08:21:23.767769+0000","last_unstale":"2024-09-17T08:21:26.777100+0000","last_undegraded":"2024-09-17T08:21:26.777100+0000","last_fullsized":"2024-09-17T08:21:26.777100+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:15.462728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953777+0000","last_change":"2024-09-17T08:21:25.778567+0000","last_active":"2024-09-17T08:21:26.953777+0000","last_peered":"2024-09-17T08:21:26.953777+0000","last_clean":"2024-09-17T08:21:26.953777+0000","last_became_active":"2024-09-17T08:21:25.777796+0000","last_became_peered":"2024-09-17T08:21:25.777796+0000","last_unstale":"2024-09-17T08:21:26.953777+0000","last_undegraded":"2024-09-17T08:21:26.953777+0000","last_fullsized":"2024-09-17T08:21:26.953777+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:28.766660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"30'11","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.104980+0000","last_change":"2024-09-17T08:21:21.760425+0000","last_active":"2024-09-17T08:21:27.104980+0000","last_peered":"2024-09-17T08:21:27.104980+0000","last_clean":"2024-09-17T08:21:27.104980+0000","last_became_active":"2024-09-17T08:21:21.760131+0000","last_became_peered":"2024-09-17T08:21:21.760131+0000","last_unstale":"2024-09-17T08:21:27.104980+0000","last_undegraded":"2024-09-17T08:21:27.104980+0000","last_fullsized":"2024-09-17T08:21:27.104980+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:43:06.079541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"23'2","reported_seq":25,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.886368+0000","last_change":"2024-09-17T08:21:19.753602+0000","last_active":"2024-09-17T08:21:26.886368+0000","last_peered":"2024-09-17T08:21:26.886368+0000","last_clean":"2024-09-17T08:21:26.886368+0000","last_became_active":"2024-09-17T08:21:19.753117+0000","last_became_peered":"2024-09-17T08:21:19.753117+0000","last_unstale":"2024-09-17T08:21:26.886368+0000","last_undegraded":"2024-09-17T08:21:26.886368+0000","last_fullsized":"2024-09-17T08:21:26.886368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:56:05.089959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"30'8","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T08:21:47.899764+0000","last_change":"2024-09-17T08:21:23.763581+0000","last_active":"2024-09-17T08:21:47.899764+0000","last_peered":"2024-09-17T08:21:47.899764+0000","last_clean":"2024-09-17T08:21:47.899764+0000","last_became_active":"2024-09-17T08:21:23.763304+0000","last_became_peered":"2024-09-17T08:21:23.763304+0000","last_unstale":"2024-09-17T08:21:47.899764+0000","last_undegraded":"2024-09-17T08:21:47.899764+0000","last_fullsized":"2024-09-17T08:21:47.899764+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:04.870366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.775999+0000","last_change":"2024-09-17T08:21:25.778390+0000","last_active":"2024-09-17T08:21:26.775999+0000","last_peered":"2024-09-17T08:21:26.775999+0000","last_clean":"2024-09-17T08:21:26.775999+0000","last_became_active":"2024-09-17T08:21:25.777933+0000","last_became_peered":"2024-09-17T08:21:25.777933+0000","last_unstale":"2024-09-17T08:21:26.775999+0000","last_undegraded":"2024-09-17T08:21:26.775999+0000","last_fullsized":"2024-09-17T08:21:26.775999+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:31.568262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"30'11","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.112348+0000","last_change":"2024-09-17T08:21:21.760651+0000","last_active":"2024-09-17T08:21:27.112348+0000","last_peered":"2024-09-17T08:21:27.112348+0000","last_clean":"2024-09-17T08:21:27.112348+0000","last_became_active":"2024-09-17T08:21:21.760336+0000","last_became_peered":"2024-09-17T08:21:21.760336+0000","last_unstale":"2024-09-17T08:21:27.112348+0000","last_undegraded":"2024-09-17T08:21:27.112348+0000","last_fullsized":"2024-09-17T08:21:27.112348+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:58.553273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776209+0000","last_change":"2024-09-17T08:21:19.753200+0000","last_active":"2024-09-17T08:21:26.776209+0000","last_peered":"2024-09-17T08:21:26.776209+0000","last_clean":"2024-09-17T08:21:26.776209+0000","last_became_active":"2024-09-17T08:21:19.752610+0000","last_became_peered":"2024-09-17T08:21:19.752610+0000","last_unstale":"2024-09-17T08:21:26.776209+0000","last_undegraded":"2024-09-17T08:21:26.776209+0000","last_fullsized":"2024-09-17T08:21:26.776209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:04:38.663894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777757+0000","last_change":"2024-09-17T08:21:23.768110+0000","last_active":"2024-09-17T08:21:26.777757+0000","last_peered":"2024-09-17T08:21:26.777757+0000","last_clean":"2024-09-17T08:21:26.777757+0000","last_became_active":"2024-09-17T08:21:23.767776+0000","last_became_peered":"2024-09-17T08:21:23.767776+0000","last_unstale":"2024-09-17T08:21:26.777757+0000","last_undegraded":"2024-09-17T08:21:26.777757+0000","last_fullsized":"2024-09-17T08:21:26.777757+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:55.194976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:29.452969+0000","last_change":"2024-09-17T08:21:25.781994+0000","last_active":"2024-09-17T08:21:29.452969+0000","last_peered":"2024-09-17T08:21:29.452969+0000","last_clean":"2024-09-17T08:21:29.452969+0000","last_became_active":"2024-09-17T08:21:25.781704+0000","last_became_peered":"2024-09-17T08:21:25.781704+0000","last_unstale":"2024-09-17T08:21:29.452969+0000","last_undegraded":"2024-09-17T08:21:29.452969+0000","last_fullsized":"2024-09-17T08:21:29.452969+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:09.488605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"30'4","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.931210+0000","last_change":"2024-09-17T08:21:21.760993+0000","last_active":"2024-09-17T08:21:26.931210+0000","last_peered":"2024-09-17T08:21:26.931210+0000","last_clean":"2024-09-17T08:21:26.931210+0000","last_became_active":"2024-09-17T08:21:21.760529+0000","last_became_peered":"2024-09-17T08:21:21.760529+0000","last_unstale":"2024-09-17T08:21:26.931210+0000","last_undegraded":"2024-09-17T08:21:26.931210+0000","last_fullsized":"2024-09-17T08:21:26.931210+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:25.529349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.778825+0000","last_change":"2024-09-17T08:21:19.754084+0000","last_active":"2024-09-17T08:21:26.778825+0000","last_peered":"2024-09-17T08:21:26.778825+0000","last_clean":"2024-09-17T08:21:26.778825+0000","last_became_active":"2024-09-17T08:21:19.753350+0000","last_became_peered":"2024-09-17T08:21:19.753350+0000","last_unstale":"2024-09-17T08:21:26.778825+0000","last_undegraded":"2024-09-17T08:21:26.778825+0000","last_fullsized":"2024-09-17T08:21:26.778825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:30:21.712101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777678+0000","last_change":"2024-09-17T08:21:23.767933+0000","last_active":"2024-09-17T08:21:26.777678+0000","last_peered":"2024-09-17T08:21:26.777678+0000","last_clean":"2024-09-17T08:21:26.777678+0000","last_became_active":"2024-09-17T08:21:23.767478+0000","last_became_peered":"2024-09-17T08:21:23.767478+0000","last_unstale":"2024-09-17T08:21:26.777678+0000","last_undegraded":"2024-09-17T08:21:26.777678+0000","last_fullsized":"2024-09-17T08:21:26.777678+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:00:42.403121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.954034+0000","last_change":"2024-09-17T08:21:25.778517+0000","last_active":"2024-09-17T08:21:26.954034+0000","last_peered":"2024-09-17T08:21:26.954034+0000","last_clean":"2024-09-17T08:21:26.954034+0000","last_became_active":"2024-09-17T08:21:25.778044+0000","last_became_peered":"2024-09-17T08:21:25.778044+0000","last_unstale":"2024-09-17T08:21:26.954034+0000","last_undegraded":"2024-09-17T08:21:26.954034+0000","last_fullsized":"2024-09-17T08:21:26.954034+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:54.886050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"30'11","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.080636+0000","last_change":"2024-09-17T08:21:21.760413+0000","last_active":"2024-09-17T08:21:27.080636+0000","last_peered":"2024-09-17T08:21:27.080636+0000","last_clean":"2024-09-17T08:21:27.080636+0000","last_became_active":"2024-09-17T08:21:21.760077+0000","last_became_peered":"2024-09-17T08:21:21.760077+0000","last_unstale":"2024-09-17T08:21:27.080636+0000","last_undegraded":"2024-09-17T08:21:27.080636+0000","last_fullsized":"2024-09-17T08:21:27.080636+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:17:06.909689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776528+0000","last_change":"2024-09-17T08:21:19.753028+0000","last_active":"2024-09-17T08:21:26.776528+0000","last_peered":"2024-09-17T08:21:26.776528+0000","last_clean":"2024-09-17T08:21:26.776528+0000","last_became_active":"2024-09-17T08:21:19.752711+0000","last_became_peered":"2024-09-17T08:21:19.752711+0000","last_unstale":"2024-09-17T08:21:26.776528+0000","last_undegraded":"2024-09-17T08:21:26.776528+0000","last_fullsized":"2024-09-17T08:21:26.776528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:14:49.844405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"30'8","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T08:21:47.899754+0000","last_change":"2024-09-17T08:21:23.767878+0000","last_active":"2024-09-17T08:21:47.899754+0000","last_peered":"2024-09-17T08:21:47.899754+0000","last_clean":"2024-09-17T08:21:47.899754+0000","last_became_active":"2024-09-17T08:21:23.767597+0000","last_became_peered":"2024-09-17T08:21:23.767597+0000","last_unstale":"2024-09-17T08:21:47.899754+0000","last_undegraded":"2024-09-17T08:21:47.899754+0000","last_fullsized":"2024-09-17T08:21:47.899754+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:36.067005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953999+0000","last_change":"2024-09-17T08:21:25.778812+0000","last_active":"2024-09-17T08:21:26.953999+0000","last_peered":"2024-09-17T08:21:26.953999+0000","last_clean":"2024-09-17T08:21:26.953999+0000","last_became_active":"2024-09-17T08:21:25.778288+0000","last_became_peered":"2024-09-17T08:21:25.778288+0000","last_unstale":"2024-09-17T08:21:26.953999+0000","last_undegraded":"2024-09-17T08:21:26.953999+0000","last_fullsized":"2024-09-17T08:21:26.953999+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:03:38.851264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.008513+0000","last_change":"2024-09-17T08:21:21.760586+0000","last_active":"2024-09-17T08:21:27.008513+0000","last_peered":"2024-09-17T08:21:27.008513+0000","last_clean":"2024-09-17T08:21:27.008513+0000","last_became_active":"2024-09-17T08:21:21.760206+0000","last_became_peered":"2024-09-17T08:21:21.760206+0000","last_unstale":"2024-09-17T08:21:27.008513+0000","last_undegraded":"2024-09-17T08:21:27.008513+0000","last_fullsized":"2024-09-17T08:21:27.008513+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:46:00.593846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776787+0000","last_change":"2024-09-17T08:21:19.753810+0000","last_active":"2024-09-17T08:21:26.776787+0000","last_peered":"2024-09-17T08:21:26.776787+0000","last_clean":"2024-09-17T08:21:26.776787+0000","last_became_active":"2024-09-17T08:21:19.753338+0000","last_became_peered":"2024-09-17T08:21:19.753338+0000","last_unstale":"2024-09-17T08:21:26.776787+0000","last_undegraded":"2024-09-17T08:21:26.776787+0000","last_fullsized":"2024-09-17T08:21:26.776787+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:46:49.961395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"30'8","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T08:21:47.899733+0000","last_change":"2024-09-17T08:21:23.764798+0000","last_active":"2024-09-17T08:21:47.899733+0000","last_peered":"2024-09-17T08:21:47.899733+0000","last_clean":"2024-09-17T08:21:47.899733+0000","last_became_active":"2024-09-17T08:21:23.764557+0000","last_became_peered":"2024-09-17T08:21:23.764557+0000","last_unstale":"2024-09-17T08:21:47.899733+0000","last_undegraded":"2024-09-17T08:21:47.899733+0000","last_fullsized":"2024-09-17T08:21:47.899733+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:04:13.207460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:29.452683+0000","last_change":"2024-09-17T08:21:25.780002+0000","last_active":"2024-09-17T08:21:29.452683+0000","last_peered":"2024-09-17T08:21:29.452683+0000","last_clean":"2024-09-17T08:21:29.452683+0000","last_became_active":"2024-09-17T08:21:25.779697+0000","last_became_peered":"2024-09-17T08:21:25.779697+0000","last_unstale":"2024-09-17T08:21:29.452683+0000","last_undegraded":"2024-09-17T08:21:29.452683+0000","last_fullsized":"2024-09-17T08:21:29.452683+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:46.975857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.092201+0000","last_change":"2024-09-17T08:21:21.760661+0000","last_active":"2024-09-17T08:21:27.092201+0000","last_peered":"2024-09-17T08:21:27.092201+0000","last_clean":"2024-09-17T08:21:27.092201+0000","last_became_active":"2024-09-17T08:21:21.759994+0000","last_became_peered":"2024-09-17T08:21:21.759994+0000","last_unstale":"2024-09-17T08:21:27.092201+0000","last_undegraded":"2024-09-17T08:21:27.092201+0000","last_fullsized":"2024-09-17T08:21:27.092201+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:34.464810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.780483+0000","last_change":"2024-09-17T08:21:19.753492+0000","last_active":"2024-09-17T08:21:26.780483+0000","last_peered":"2024-09-17T08:21:26.780483+0000","last_clean":"2024-09-17T08:21:26.780483+0000","last_became_active":"2024-09-17T08:21:19.753104+0000","last_became_peered":"2024-09-17T08:21:19.753104+0000","last_unstale":"2024-09-17T08:21:26.780483+0000","last_undegraded":"2024-09-17T08:21:26.780483+0000","last_fullsized":"2024-09-17T08:21:26.780483+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:48.506654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776214+0000","last_change":"2024-09-17T08:21:23.766953+0000","last_active":"2024-09-17T08:21:26.776214+0000","last_peered":"2024-09-17T08:21:26.776214+0000","last_clean":"2024-09-17T08:21:26.776214+0000","last_became_active":"2024-09-17T08:21:23.766357+0000","last_became_peered":"2024-09-17T08:21:23.766357+0000","last_unstale":"2024-09-17T08:21:26.776214+0000","last_undegraded":"2024-09-17T08:21:26.776214+0000","last_fullsized":"2024-09-17T08:21:26.776214+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:15:03.591049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776640+0000","last_change":"2024-09-17T08:21:25.778423+0000","last_active":"2024-09-17T08:21:26.776640+0000","last_peered":"2024-09-17T08:21:26.776640+0000","last_clean":"2024-09-17T08:21:26.776640+0000","last_became_active":"2024-09-17T08:21:25.777964+0000","last_became_peered":"2024-09-17T08:21:25.777964+0000","last_unstale":"2024-09-17T08:21:26.776640+0000","last_undegraded":"2024-09-17T08:21:26.776640+0000","last_fullsized":"2024-09-17T08:21:26.776640+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:35.618652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"30'10","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.089837+0000","last_change":"2024-09-17T08:21:21.761279+0000","last_active":"2024-09-17T08:21:27.089837+0000","last_peered":"2024-09-17T08:21:27.089837+0000","last_clean":"2024-09-17T08:21:27.089837+0000","last_became_active":"2024-09-17T08:21:21.760825+0000","last_became_peered":"2024-09-17T08:21:21.760825+0000","last_unstale":"2024-09-17T08:21:27.089837+0000","last_undegraded":"2024-09-17T08:21:27.089837+0000","last_fullsized":"2024-09-17T08:21:27.089837+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:18:14.553324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"23'1","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.886712+0000","last_change":"2024-09-17T08:21:19.755088+0000","last_active":"2024-09-17T08:21:26.886712+0000","last_peered":"2024-09-17T08:21:26.886712+0000","last_clean":"2024-09-17T08:21:26.886712+0000","last_became_active":"2024-09-17T08:21:19.754831+0000","last_became_peered":"2024-09-17T08:21:19.754831+0000","last_unstale":"2024-09-17T08:21:26.886712+0000","last_undegraded":"2024-09-17T08:21:26.886712+0000","last_fullsized":"2024-09-17T08:21:26.886712+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:00:46.174385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953904+0000","last_change":"2024-09-17T08:21:23.771371+0000","last_active":"2024-09-17T08:21:26.953904+0000","last_peered":"2024-09-17T08:21:26.953904+0000","last_clean":"2024-09-17T08:21:26.953904+0000","last_became_active":"2024-09-17T08:21:23.771076+0000","last_became_peered":"2024-09-17T08:21:23.771076+0000","last_unstale":"2024-09-17T08:21:26.953904+0000","last_undegraded":"2024-09-17T08:21:26.953904+0000","last_fullsized":"2024-09-17T08:21:26.953904+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:46.042493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777979+0000","last_change":"2024-09-17T08:21:25.779255+0000","last_active":"2024-09-17T08:21:26.777979+0000","last_peered":"2024-09-17T08:21:26.777979+0000","last_clean":"2024-09-17T08:21:26.777979+0000","last_became_active":"2024-09-17T08:21:25.778791+0000","last_became_peered":"2024-09-17T08:21:25.778791+0000","last_unstale":"2024-09-17T08:21:26.777979+0000","last_undegraded":"2024-09-17T08:21:26.777979+0000","last_fullsized":"2024-09-17T08:21:26.777979+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:13.928809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"30'6","reported_seq":25,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.071447+0000","last_change":"2024-09-17T08:21:21.765324+0000","last_active":"2024-09-17T08:21:27.071447+0000","last_peered":"2024-09-17T08:21:27.071447+0000","last_clean":"2024-09-17T08:21:27.071447+0000","last_became_active":"2024-09-17T08:21:21.764774+0000","last_became_peered":"2024-09-17T08:21:21.764774+0000","last_unstale":"2024-09-17T08:21:27.071447+0000","last_undegraded":"2024-09-17T08:21:27.071447+0000","last_fullsized":"2024-09-17T08:21:27.071447+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:06.870820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.780359+0000","last_change":"2024-09-17T08:21:19.757862+0000","last_active":"2024-09-17T08:21:26.780359+0000","last_peered":"2024-09-17T08:21:26.780359+0000","last_clean":"2024-09-17T08:21:26.780359+0000","last_became_active":"2024-09-17T08:21:19.757553+0000","last_became_peered":"2024-09-17T08:21:19.757553+0000","last_unstale":"2024-09-17T08:21:26.780359+0000","last_undegraded":"2024-09-17T08:21:26.780359+0000","last_fullsized":"2024-09-17T08:21:26.780359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:46:02.143139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.775752+0000","last_change":"2024-09-17T08:21:23.768657+0000","last_active":"2024-09-17T08:21:26.775752+0000","last_peered":"2024-09-17T08:21:26.775752+0000","last_clean":"2024-09-17T08:21:26.775752+0000","last_became_active":"2024-09-17T08:21:23.768357+0000","last_became_peered":"2024-09-17T08:21:23.768357+0000","last_unstale":"2024-09-17T08:21:26.775752+0000","last_undegraded":"2024-09-17T08:21:26.775752+0000","last_fullsized":"2024-09-17T08:21:26.775752+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:27:14.670258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953768+0000","last_change":"2024-09-17T08:21:25.778156+0000","last_active":"2024-09-17T08:21:26.953768+0000","last_peered":"2024-09-17T08:21:26.953768+0000","last_clean":"2024-09-17T08:21:26.953768+0000","last_became_active":"2024-09-17T08:21:25.777699+0000","last_became_peered":"2024-09-17T08:21:25.777699+0000","last_unstale":"2024-09-17T08:21:26.953768+0000","last_undegraded":"2024-09-17T08:21:26.953768+0000","last_fullsized":"2024-09-17T08:21:26.953768+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:46:57.612816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.944689+0000","last_change":"2024-09-17T08:21:21.763281+0000","last_active":"2024-09-17T08:21:26.944689+0000","last_peered":"2024-09-17T08:21:26.944689+0000","last_clean":"2024-09-17T08:21:26.944689+0000","last_became_active":"2024-09-17T08:21:21.762313+0000","last_became_peered":"2024-09-17T08:21:21.762313+0000","last_unstale":"2024-09-17T08:21:26.944689+0000","last_undegraded":"2024-09-17T08:21:26.944689+0000","last_fullsized":"2024-09-17T08:21:26.944689+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:28:52.720354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.780859+0000","last_change":"2024-09-17T08:21:19.754848+0000","last_active":"2024-09-17T08:21:26.780859+0000","last_peered":"2024-09-17T08:21:26.780859+0000","last_clean":"2024-09-17T08:21:26.780859+0000","last_became_active":"2024-09-17T08:21:19.754615+0000","last_became_peered":"2024-09-17T08:21:19.754615+0000","last_unstale":"2024-09-17T08:21:26.780859+0000","last_undegraded":"2024-09-17T08:21:26.780859+0000","last_fullsized":"2024-09-17T08:21:26.780859+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:38:15.073793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777340+0000","last_change":"2024-09-17T08:21:23.766760+0000","last_active":"2024-09-17T08:21:26.777340+0000","last_peered":"2024-09-17T08:21:26.777340+0000","last_clean":"2024-09-17T08:21:26.777340+0000","last_became_active":"2024-09-17T08:21:23.766367+0000","last_became_peered":"2024-09-17T08:21:23.766367+0000","last_unstale":"2024-09-17T08:21:26.777340+0000","last_undegraded":"2024-09-17T08:21:26.777340+0000","last_fullsized":"2024-09-17T08:21:26.777340+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:32.505627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953723+0000","last_change":"2024-09-17T08:21:25.778727+0000","last_active":"2024-09-17T08:21:26.953723+0000","last_peered":"2024-09-17T08:21:26.953723+0000","last_clean":"2024-09-17T08:21:26.953723+0000","last_became_active":"2024-09-17T08:21:25.778362+0000","last_became_peered":"2024-09-17T08:21:25.778362+0000","last_unstale":"2024-09-17T08:21:26.953723+0000","last_undegraded":"2024-09-17T08:21:26.953723+0000","last_fullsized":"2024-09-17T08:21:26.953723+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:58:39.289050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.954043+0000","last_change":"2024-09-17T08:21:25.779584+0000","last_active":"2024-09-17T08:21:26.954043+0000","last_peered":"2024-09-17T08:21:26.954043+0000","last_clean":"2024-09-17T08:21:26.954043+0000","last_became_active":"2024-09-17T08:21:25.779282+0000","last_became_peered":"2024-09-17T08:21:25.779282+0000","last_unstale":"2024-09-17T08:21:26.954043+0000","last_undegraded":"2024-09-17T08:21:26.954043+0000","last_fullsized":"2024-09-17T08:21:26.954043+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:41.234351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"30'15","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.119199+0000","last_change":"2024-09-17T08:21:21.760534+0000","last_active":"2024-09-17T08:21:27.119199+0000","last_peered":"2024-09-17T08:21:27.119199+0000","last_clean":"2024-09-17T08:21:27.119199+0000","last_became_active":"2024-09-17T08:21:21.759936+0000","last_became_peered":"2024-09-17T08:21:21.759936+0000","last_unstale":"2024-09-17T08:21:27.119199+0000","last_undegraded":"2024-09-17T08:21:27.119199+0000","last_fullsized":"2024-09-17T08:21:27.119199+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:23:10.660424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776248+0000","last_change":"2024-09-17T08:21:19.753478+0000","last_active":"2024-09-17T08:21:26.776248+0000","last_peered":"2024-09-17T08:21:26.776248+0000","last_clean":"2024-09-17T08:21:26.776248+0000","last_became_active":"2024-09-17T08:21:19.753189+0000","last_became_peered":"2024-09-17T08:21:19.753189+0000","last_unstale":"2024-09-17T08:21:26.776248+0000","last_undegraded":"2024-09-17T08:21:26.776248+0000","last_fullsized":"2024-09-17T08:21:26.776248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:04.063852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777884+0000","last_change":"2024-09-17T08:21:23.768003+0000","last_active":"2024-09-17T08:21:26.777884+0000","last_peered":"2024-09-17T08:21:26.777884+0000","last_clean":"2024-09-17T08:21:26.777884+0000","last_became_active":"2024-09-17T08:21:23.767624+0000","last_became_peered":"2024-09-17T08:21:23.767624+0000","last_unstale":"2024-09-17T08:21:26.777884+0000","last_undegraded":"2024-09-17T08:21:26.777884+0000","last_fullsized":"2024-09-17T08:21:26.777884+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:07.264004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:29.452804+0000","last_change":"2024-09-17T08:21:25.781935+0000","last_active":"2024-09-17T08:21:29.452804+0000","last_peered":"2024-09-17T08:21:29.452804+0000","last_clean":"2024-09-17T08:21:29.452804+0000","last_became_active":"2024-09-17T08:21:25.781627+0000","last_became_peered":"2024-09-17T08:21:25.781627+0000","last_unstale":"2024-09-17T08:21:29.452804+0000","last_undegraded":"2024-09-17T08:21:29.452804+0000","last_fullsized":"2024-09-17T08:21:29.452804+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:04:37.550722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.910065+0000","last_change":"2024-09-17T08:21:21.762101+0000","last_active":"2024-09-17T08:21:26.910065+0000","last_peered":"2024-09-17T08:21:26.910065+0000","last_clean":"2024-09-17T08:21:26.910065+0000","last_became_active":"2024-09-17T08:21:21.761572+0000","last_became_peered":"2024-09-17T08:21:21.761572+0000","last_unstale":"2024-09-17T08:21:26.910065+0000","last_undegraded":"2024-09-17T08:21:26.910065+0000","last_fullsized":"2024-09-17T08:21:26.910065+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:09:55.503404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.775519+0000","last_change":"2024-09-17T08:21:19.754050+0000","last_active":"2024-09-17T08:21:26.775519+0000","last_peered":"2024-09-17T08:21:26.775519+0000","last_clean":"2024-09-17T08:21:26.775519+0000","last_became_active":"2024-09-17T08:21:19.753088+0000","last_became_peered":"2024-09-17T08:21:19.753088+0000","last_unstale":"2024-09-17T08:21:26.775519+0000","last_undegraded":"2024-09-17T08:21:26.775519+0000","last_fullsized":"2024-09-17T08:21:26.775519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:10.277650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'8","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T08:21:47.899689+0000","last_change":"2024-09-17T08:21:23.767416+0000","last_active":"2024-09-17T08:21:47.899689+0000","last_peered":"2024-09-17T08:21:47.899689+0000","last_clean":"2024-09-17T08:21:47.899689+0000","last_became_active":"2024-09-17T08:21:23.767108+0000","last_became_peered":"2024-09-17T08:21:23.767108+0000","last_unstale":"2024-09-17T08:21:47.899689+0000","last_undegraded":"2024-09-17T08:21:47.899689+0000","last_fullsized":"2024-09-17T08:21:47.899689+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:39.626106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.779121+0000","last_change":"2024-09-17T08:21:19.753525+0000","last_active":"2024-09-17T08:21:26.779121+0000","last_peered":"2024-09-17T08:21:26.779121+0000","last_clean":"2024-09-17T08:21:26.779121+0000","last_became_active":"2024-09-17T08:21:19.752975+0000","last_became_peered":"2024-09-17T08:21:19.752975+0000","last_unstale":"2024-09-17T08:21:26.779121+0000","last_undegraded":"2024-09-17T08:21:26.779121+0000","last_fullsized":"2024-09-17T08:21:26.779121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:20:29.857759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"30'5","reported_seq":26,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.992232+0000","last_change":"2024-09-17T08:21:21.760487+0000","last_active":"2024-09-17T08:21:26.992232+0000","last_peered":"2024-09-17T08:21:26.992232+0000","last_clean":"2024-09-17T08:21:26.992232+0000","last_became_active":"2024-09-17T08:21:21.759984+0000","last_became_peered":"2024-09-17T08:21:21.759984+0000","last_unstale":"2024-09-17T08:21:26.992232+0000","last_undegraded":"2024-09-17T08:21:26.992232+0000","last_fullsized":"2024-09-17T08:21:26.992232+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:42:48.116537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.778667+0000","last_change":"2024-09-17T08:21:23.768298+0000","last_active":"2024-09-17T08:21:26.778667+0000","last_peered":"2024-09-17T08:21:26.778667+0000","last_clean":"2024-09-17T08:21:26.778667+0000","last_became_active":"2024-09-17T08:21:23.768009+0000","last_became_peered":"2024-09-17T08:21:23.768009+0000","last_unstale":"2024-09-17T08:21:26.778667+0000","last_undegraded":"2024-09-17T08:21:26.778667+0000","last_fullsized":"2024-09-17T08:21:26.778667+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:38.554959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:29.452756+0000","last_change":"2024-09-17T08:21:25.780375+0000","last_active":"2024-09-17T08:21:29.452756+0000","last_peered":"2024-09-17T08:21:29.452756+0000","last_clean":"2024-09-17T08:21:29.452756+0000","last_became_active":"2024-09-17T08:21:25.779669+0000","last_became_peered":"2024-09-17T08:21:25.779669+0000","last_unstale":"2024-09-17T08:21:29.452756+0000","last_undegraded":"2024-09-17T08:21:29.452756+0000","last_fullsized":"2024-09-17T08:21:29.452756+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:29.469839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777234+0000","last_change":"2024-09-17T08:21:19.753621+0000","last_active":"2024-09-17T08:21:26.777234+0000","last_peered":"2024-09-17T08:21:26.777234+0000","last_clean":"2024-09-17T08:21:26.777234+0000","last_became_active":"2024-09-17T08:21:19.752767+0000","last_became_peered":"2024-09-17T08:21:19.752767+0000","last_unstale":"2024-09-17T08:21:26.777234+0000","last_undegraded":"2024-09-17T08:21:26.777234+0000","last_fullsized":"2024-09-17T08:21:26.777234+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:01.863983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.015134+0000","last_change":"2024-09-17T08:21:21.760582+0000","last_active":"2024-09-17T08:21:27.015134+0000","last_peered":"2024-09-17T08:21:27.015134+0000","last_clean":"2024-09-17T08:21:27.015134+0000","last_became_active":"2024-09-17T08:21:21.760072+0000","last_became_peered":"2024-09-17T08:21:21.760072+0000","last_unstale":"2024-09-17T08:21:27.015134+0000","last_undegraded":"2024-09-17T08:21:27.015134+0000","last_fullsized":"2024-09-17T08:21:27.015134+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:00.017875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.778534+0000","last_change":"2024-09-17T08:21:23.764650+0000","last_active":"2024-09-17T08:21:26.778534+0000","last_peered":"2024-09-17T08:21:26.778534+0000","last_clean":"2024-09-17T08:21:26.778534+0000","last_became_active":"2024-09-17T08:21:23.764379+0000","last_became_peered":"2024-09-17T08:21:23.764379+0000","last_unstale":"2024-09-17T08:21:26.778534+0000","last_undegraded":"2024-09-17T08:21:26.778534+0000","last_fullsized":"2024-09-17T08:21:26.778534+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:01.400596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953958+0000","last_change":"2024-09-17T08:21:25.777731+0000","last_active":"2024-09-17T08:21:26.953958+0000","last_peered":"2024-09-17T08:21:26.953958+0000","last_clean":"2024-09-17T08:21:26.953958+0000","last_became_active":"2024-09-17T08:21:25.777427+0000","last_became_peered":"2024-09-17T08:21:25.777427+0000","last_unstale":"2024-09-17T08:21:26.953958+0000","last_undegraded":"2024-09-17T08:21:26.953958+0000","last_fullsized":"2024-09-17T08:21:26.953958+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:12:54.042496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":18,"seq":77309411341,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32132,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708924,"statfs":{"total":95990841344,"available":95957938176,"internally_reserved":0,"allocated":1044480,"data_stored":699977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32136,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708920,"statfs":{"total":95990841344,"available":95957934080,"internally_reserved":0,"allocated":1044480,"data_stored":699977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.45100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738391,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":36232,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93704824,"statfs":{"total":95990841344,"available":95953739776,"internally_reserved":0,"allocated":1044480,"data_stored":699977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.60799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:22:01.651 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:22:01.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:01 smithi203 ceph-mon[30259]: from='client.14496 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:22:01.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:01 smithi193 ceph-mon[30002]: from='client.14496 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:22:02.631 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:02 smithi047 ceph-mon[25866]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:02.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:02 smithi203 ceph-mon[30259]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:02.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:02 smithi193 ceph-mon[30002]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:03.813 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:04.745 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:04 smithi047 ceph-mon[25866]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:04.745 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:22:04 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: ::ffff:172.21.15.193 - - [17/Sep/2024:08:22:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:22:04.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:04 smithi203 ceph-mon[30259]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:04.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:04 smithi193 ceph-mon[30002]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:06.444 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:22:06.446 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-09-17T08:22:06.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:06 smithi203 ceph-mon[30259]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:06.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:06 smithi047 ceph-mon[25866]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:06.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:06 smithi193 ceph-mon[30002]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:07.155 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:22:06 smithi047 podman[55802]: 2024-09-17 08:22:06.895085331 +0000 UTC m=+1.309149502 container create c04cdbc35cb1b475ee1d4c6d6fb4e73d7774e658dac4658f8252779c800e90af (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:22:07.155 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:22:07 smithi047 podman[55802]: 2024-09-17 08:22:07.155538838 +0000 UTC m=+1.569602990 container init c04cdbc35cb1b475ee1d4c6d6fb4e73d7774e658dac4658f8252779c800e90af (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:22:07.449 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: [17/Sep/2024:08:22:07] ENGINE Bus STOPPING 2024-09-17T08:22:07.449 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:22:07 smithi047 podman[55802]: 2024-09-17 08:22:07.160589442 +0000 UTC m=+1.574653583 container start c04cdbc35cb1b475ee1d4c6d6fb4e73d7774e658dac4658f8252779c800e90af (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:22:07.449 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-alertmanager-a[55876]: ts=2024-09-17T08:22:07.173Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-17T08:22:07.449 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-alertmanager-a[55876]: ts=2024-09-17T08:22:07.173Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-17T08:22:07.449 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-alertmanager-a[55876]: ts=2024-09-17T08:22:07.174Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.47 port=9094 2024-09-17T08:22:07.450 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-alertmanager-a[55876]: ts=2024-09-17T08:22:07.175Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-17T08:22:07.450 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-alertmanager-a[55876]: ts=2024-09-17T08:22:07.198Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-17T08:22:07.450 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-alertmanager-a[55876]: ts=2024-09-17T08:22:07.199Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-17T08:22:07.450 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-alertmanager-a[55876]: ts=2024-09-17T08:22:07.202Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-17T08:22:07.450 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-alertmanager-a[55876]: ts=2024-09-17T08:22:07.202Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-17T08:22:07.450 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:22:07 smithi047 bash[55802]: c04cdbc35cb1b475ee1d4c6d6fb4e73d7774e658dac4658f8252779c800e90af 2024-09-17T08:22:07.450 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:22:07 smithi047 systemd[1]: Started Ceph alertmanager.a for 3a66518a-74cd-11ef-bceb-c7b262605968. 2024-09-17T08:22:07.725 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: [17/Sep/2024:08:22:07] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T08:22:07.725 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: [17/Sep/2024:08:22:07] ENGINE Bus STOPPED 2024-09-17T08:22:07.725 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: [17/Sep/2024:08:22:07] ENGINE Bus STARTING 2024-09-17T08:22:07.725 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: [17/Sep/2024:08:22:07] ENGINE Serving on http://:::9283 2024-09-17T08:22:07.725 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: [17/Sep/2024:08:22:07] ENGINE Bus STARTED 2024-09-17T08:22:07.725 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-mon[25866]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:22:07.725 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:07.725 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:07.726 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:07.726 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:07.726 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:07.726 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:07.726 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T08:22:07.726 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:07 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:07.747 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-09-17T08:22:05.132733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595838,"num_objects":193,"num_object_clones":0,"num_object_copies":579,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":193,"num_whiteouts":0,"num_read":962,"num_read_kb":809,"num_write":691,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":669,"ondisk_log_size":669,"up":387,"acting":387,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":387,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":281223168,"kb_used":100500,"kb_used_data":3060,"kb_used_omap":4,"kb_used_meta":97403,"kb_avail":281122668,"statfs":{"total":287972524032,"available":287869612032,"internally_reserved":0,"allocated":3133440,"data_stored":2099931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4775,"internal_metadata":99741017},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002743"},"pg_stats":[{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777186+0000","last_change":"2024-09-17T08:21:19.753809+0000","last_active":"2024-09-17T08:21:26.777186+0000","last_peered":"2024-09-17T08:21:26.777186+0000","last_clean":"2024-09-17T08:21:26.777186+0000","last_became_active":"2024-09-17T08:21:19.753364+0000","last_became_peered":"2024-09-17T08:21:19.753364+0000","last_unstale":"2024-09-17T08:21:26.777186+0000","last_undegraded":"2024-09-17T08:21:26.777186+0000","last_fullsized":"2024-09-17T08:21:26.777186+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:48:17.211239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'10","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.107482+0000","last_change":"2024-09-17T08:21:21.761298+0000","last_active":"2024-09-17T08:21:27.107482+0000","last_peered":"2024-09-17T08:21:27.107482+0000","last_clean":"2024-09-17T08:21:27.107482+0000","last_became_active":"2024-09-17T08:21:21.760143+0000","last_became_peered":"2024-09-17T08:21:21.760143+0000","last_unstale":"2024-09-17T08:21:27.107482+0000","last_undegraded":"2024-09-17T08:21:27.107482+0000","last_fullsized":"2024-09-17T08:21:27.107482+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:26:22.799002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776291+0000","last_change":"2024-09-17T08:21:23.766760+0000","last_active":"2024-09-17T08:21:26.776291+0000","last_peered":"2024-09-17T08:21:26.776291+0000","last_clean":"2024-09-17T08:21:26.776291+0000","last_became_active":"2024-09-17T08:21:23.766468+0000","last_became_peered":"2024-09-17T08:21:23.766468+0000","last_unstale":"2024-09-17T08:21:26.776291+0000","last_undegraded":"2024-09-17T08:21:26.776291+0000","last_fullsized":"2024-09-17T08:21:26.776291+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:15:56.495634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953709+0000","last_change":"2024-09-17T08:21:25.778378+0000","last_active":"2024-09-17T08:21:26.953709+0000","last_peered":"2024-09-17T08:21:26.953709+0000","last_clean":"2024-09-17T08:21:26.953709+0000","last_became_active":"2024-09-17T08:21:25.777985+0000","last_became_peered":"2024-09-17T08:21:25.777985+0000","last_unstale":"2024-09-17T08:21:26.953709+0000","last_undegraded":"2024-09-17T08:21:26.953709+0000","last_fullsized":"2024-09-17T08:21:26.953709+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:57.441771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777557+0000","last_change":"2024-09-17T08:21:19.754820+0000","last_active":"2024-09-17T08:21:26.777557+0000","last_peered":"2024-09-17T08:21:26.777557+0000","last_clean":"2024-09-17T08:21:26.777557+0000","last_became_active":"2024-09-17T08:21:19.753725+0000","last_became_peered":"2024-09-17T08:21:19.753725+0000","last_unstale":"2024-09-17T08:21:26.777557+0000","last_undegraded":"2024-09-17T08:21:26.777557+0000","last_fullsized":"2024-09-17T08:21:26.777557+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:59:38.573365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"30'11","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.114793+0000","last_change":"2024-09-17T08:21:21.764982+0000","last_active":"2024-09-17T08:21:27.114793+0000","last_peered":"2024-09-17T08:21:27.114793+0000","last_clean":"2024-09-17T08:21:27.114793+0000","last_became_active":"2024-09-17T08:21:21.764654+0000","last_became_peered":"2024-09-17T08:21:21.764654+0000","last_unstale":"2024-09-17T08:21:27.114793+0000","last_undegraded":"2024-09-17T08:21:27.114793+0000","last_fullsized":"2024-09-17T08:21:27.114793+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:03:06.826760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953747+0000","last_change":"2024-09-17T08:21:23.768701+0000","last_active":"2024-09-17T08:21:26.953747+0000","last_peered":"2024-09-17T08:21:26.953747+0000","last_clean":"2024-09-17T08:21:26.953747+0000","last_became_active":"2024-09-17T08:21:23.767948+0000","last_became_peered":"2024-09-17T08:21:23.767948+0000","last_unstale":"2024-09-17T08:21:26.953747+0000","last_undegraded":"2024-09-17T08:21:26.953747+0000","last_fullsized":"2024-09-17T08:21:26.953747+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:39.755523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:29.452509+0000","last_change":"2024-09-17T08:21:25.775451+0000","last_active":"2024-09-17T08:21:29.452509+0000","last_peered":"2024-09-17T08:21:29.452509+0000","last_clean":"2024-09-17T08:21:29.452509+0000","last_became_active":"2024-09-17T08:21:25.775174+0000","last_became_peered":"2024-09-17T08:21:25.775174+0000","last_unstale":"2024-09-17T08:21:29.452509+0000","last_undegraded":"2024-09-17T08:21:29.452509+0000","last_fullsized":"2024-09-17T08:21:29.452509+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:53.411212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777721+0000","last_change":"2024-09-17T08:21:19.754950+0000","last_active":"2024-09-17T08:21:26.777721+0000","last_peered":"2024-09-17T08:21:26.777721+0000","last_clean":"2024-09-17T08:21:26.777721+0000","last_became_active":"2024-09-17T08:21:19.754718+0000","last_became_peered":"2024-09-17T08:21:19.754718+0000","last_unstale":"2024-09-17T08:21:26.777721+0000","last_undegraded":"2024-09-17T08:21:26.777721+0000","last_fullsized":"2024-09-17T08:21:26.777721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:02:17.791289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"30'15","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.047608+0000","last_change":"2024-09-17T08:21:21.763471+0000","last_active":"2024-09-17T08:21:27.047608+0000","last_peered":"2024-09-17T08:21:27.047608+0000","last_clean":"2024-09-17T08:21:27.047608+0000","last_became_active":"2024-09-17T08:21:21.762999+0000","last_became_peered":"2024-09-17T08:21:21.762999+0000","last_unstale":"2024-09-17T08:21:27.047608+0000","last_undegraded":"2024-09-17T08:21:27.047608+0000","last_fullsized":"2024-09-17T08:21:27.047608+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:48.972836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953913+0000","last_change":"2024-09-17T08:21:23.768390+0000","last_active":"2024-09-17T08:21:26.953913+0000","last_peered":"2024-09-17T08:21:26.953913+0000","last_clean":"2024-09-17T08:21:26.953913+0000","last_became_active":"2024-09-17T08:21:23.767529+0000","last_became_peered":"2024-09-17T08:21:23.767529+0000","last_unstale":"2024-09-17T08:21:26.953913+0000","last_undegraded":"2024-09-17T08:21:26.953913+0000","last_fullsized":"2024-09-17T08:21:26.953913+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:17:44.539844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776409+0000","last_change":"2024-09-17T08:21:25.778355+0000","last_active":"2024-09-17T08:21:26.776409+0000","last_peered":"2024-09-17T08:21:26.776409+0000","last_clean":"2024-09-17T08:21:26.776409+0000","last_became_active":"2024-09-17T08:21:25.777977+0000","last_became_peered":"2024-09-17T08:21:25.777977+0000","last_unstale":"2024-09-17T08:21:26.776409+0000","last_undegraded":"2024-09-17T08:21:26.776409+0000","last_fullsized":"2024-09-17T08:21:26.776409+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:48:13.098229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777610+0000","last_change":"2024-09-17T08:21:19.753531+0000","last_active":"2024-09-17T08:21:26.777610+0000","last_peered":"2024-09-17T08:21:26.777610+0000","last_clean":"2024-09-17T08:21:26.777610+0000","last_became_active":"2024-09-17T08:21:19.753234+0000","last_became_peered":"2024-09-17T08:21:19.753234+0000","last_unstale":"2024-09-17T08:21:26.777610+0000","last_undegraded":"2024-09-17T08:21:26.777610+0000","last_fullsized":"2024-09-17T08:21:26.777610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:06.954989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"30'12","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.121506+0000","last_change":"2024-09-17T08:21:21.760540+0000","last_active":"2024-09-17T08:21:27.121506+0000","last_peered":"2024-09-17T08:21:27.121506+0000","last_clean":"2024-09-17T08:21:27.121506+0000","last_became_active":"2024-09-17T08:21:21.760259+0000","last_became_peered":"2024-09-17T08:21:21.760259+0000","last_unstale":"2024-09-17T08:21:27.121506+0000","last_undegraded":"2024-09-17T08:21:27.121506+0000","last_fullsized":"2024-09-17T08:21:27.121506+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:08:37.299230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953830+0000","last_change":"2024-09-17T08:21:23.767741+0000","last_active":"2024-09-17T08:21:26.953830+0000","last_peered":"2024-09-17T08:21:26.953830+0000","last_clean":"2024-09-17T08:21:26.953830+0000","last_became_active":"2024-09-17T08:21:23.767269+0000","last_became_peered":"2024-09-17T08:21:23.767269+0000","last_unstale":"2024-09-17T08:21:26.953830+0000","last_undegraded":"2024-09-17T08:21:26.953830+0000","last_fullsized":"2024-09-17T08:21:26.953830+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:25:14.349907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953868+0000","last_change":"2024-09-17T08:21:25.777936+0000","last_active":"2024-09-17T08:21:26.953868+0000","last_peered":"2024-09-17T08:21:26.953868+0000","last_clean":"2024-09-17T08:21:26.953868+0000","last_became_active":"2024-09-17T08:21:25.777140+0000","last_became_peered":"2024-09-17T08:21:25.777140+0000","last_unstale":"2024-09-17T08:21:26.953868+0000","last_undegraded":"2024-09-17T08:21:26.953868+0000","last_fullsized":"2024-09-17T08:21:26.953868+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:26:55.860102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"30'19","reported_seq":47,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.125658+0000","last_change":"2024-09-17T08:21:21.763336+0000","last_active":"2024-09-17T08:21:27.125658+0000","last_peered":"2024-09-17T08:21:27.125658+0000","last_clean":"2024-09-17T08:21:27.125658+0000","last_became_active":"2024-09-17T08:21:21.762352+0000","last_became_peered":"2024-09-17T08:21:21.762352+0000","last_unstale":"2024-09-17T08:21:27.125658+0000","last_undegraded":"2024-09-17T08:21:27.125658+0000","last_fullsized":"2024-09-17T08:21:27.125658+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:01.333731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777036+0000","last_change":"2024-09-17T08:21:19.753817+0000","last_active":"2024-09-17T08:21:26.777036+0000","last_peered":"2024-09-17T08:21:26.777036+0000","last_clean":"2024-09-17T08:21:26.777036+0000","last_became_active":"2024-09-17T08:21:19.753539+0000","last_became_peered":"2024-09-17T08:21:19.753539+0000","last_unstale":"2024-09-17T08:21:26.777036+0000","last_undegraded":"2024-09-17T08:21:26.777036+0000","last_fullsized":"2024-09-17T08:21:26.777036+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:49:29.345564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"30'8","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T08:21:52.900660+0000","last_change":"2024-09-17T08:21:23.768193+0000","last_active":"2024-09-17T08:21:52.900660+0000","last_peered":"2024-09-17T08:21:52.900660+0000","last_clean":"2024-09-17T08:21:52.900660+0000","last_became_active":"2024-09-17T08:21:23.767403+0000","last_became_peered":"2024-09-17T08:21:23.767403+0000","last_unstale":"2024-09-17T08:21:52.900660+0000","last_undegraded":"2024-09-17T08:21:52.900660+0000","last_fullsized":"2024-09-17T08:21:52.900660+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:21:00.514569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:29.452704+0000","last_change":"2024-09-17T08:21:25.781641+0000","last_active":"2024-09-17T08:21:29.452704+0000","last_peered":"2024-09-17T08:21:29.452704+0000","last_clean":"2024-09-17T08:21:29.452704+0000","last_became_active":"2024-09-17T08:21:25.781062+0000","last_became_peered":"2024-09-17T08:21:25.781062+0000","last_unstale":"2024-09-17T08:21:29.452704+0000","last_undegraded":"2024-09-17T08:21:29.452704+0000","last_fullsized":"2024-09-17T08:21:29.452704+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:04.115389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.082760+0000","last_change":"2024-09-17T08:21:21.761087+0000","last_active":"2024-09-17T08:21:27.082760+0000","last_peered":"2024-09-17T08:21:27.082760+0000","last_clean":"2024-09-17T08:21:27.082760+0000","last_became_active":"2024-09-17T08:21:21.760742+0000","last_became_peered":"2024-09-17T08:21:21.760742+0000","last_unstale":"2024-09-17T08:21:27.082760+0000","last_undegraded":"2024-09-17T08:21:27.082760+0000","last_fullsized":"2024-09-17T08:21:27.082760+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:14.224261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'1","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.876282+0000","last_change":"2024-09-17T08:21:19.753880+0000","last_active":"2024-09-17T08:21:26.876282+0000","last_peered":"2024-09-17T08:21:26.876282+0000","last_clean":"2024-09-17T08:21:26.876282+0000","last_became_active":"2024-09-17T08:21:19.753604+0000","last_became_peered":"2024-09-17T08:21:19.753604+0000","last_unstale":"2024-09-17T08:21:26.876282+0000","last_undegraded":"2024-09-17T08:21:26.876282+0000","last_fullsized":"2024-09-17T08:21:26.876282+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:39:29.877304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953948+0000","last_change":"2024-09-17T08:21:23.768481+0000","last_active":"2024-09-17T08:21:26.953948+0000","last_peered":"2024-09-17T08:21:26.953948+0000","last_clean":"2024-09-17T08:21:26.953948+0000","last_became_active":"2024-09-17T08:21:23.767793+0000","last_became_peered":"2024-09-17T08:21:23.767793+0000","last_unstale":"2024-09-17T08:21:26.953948+0000","last_undegraded":"2024-09-17T08:21:26.953948+0000","last_fullsized":"2024-09-17T08:21:26.953948+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:30.754360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.778677+0000","last_change":"2024-09-17T08:21:25.779077+0000","last_active":"2024-09-17T08:21:26.778677+0000","last_peered":"2024-09-17T08:21:26.778677+0000","last_clean":"2024-09-17T08:21:26.778677+0000","last_became_active":"2024-09-17T08:21:25.778766+0000","last_became_peered":"2024-09-17T08:21:25.778766+0000","last_unstale":"2024-09-17T08:21:26.778677+0000","last_undegraded":"2024-09-17T08:21:26.778677+0000","last_fullsized":"2024-09-17T08:21:26.778677+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:22.641911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"30'15","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.054262+0000","last_change":"2024-09-17T08:21:21.758855+0000","last_active":"2024-09-17T08:21:27.054262+0000","last_peered":"2024-09-17T08:21:27.054262+0000","last_clean":"2024-09-17T08:21:27.054262+0000","last_became_active":"2024-09-17T08:21:21.758541+0000","last_became_peered":"2024-09-17T08:21:21.758541+0000","last_unstale":"2024-09-17T08:21:27.054262+0000","last_undegraded":"2024-09-17T08:21:27.054262+0000","last_fullsized":"2024-09-17T08:21:27.054262+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:20.421507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.775423+0000","last_change":"2024-09-17T08:21:19.753409+0000","last_active":"2024-09-17T08:21:26.775423+0000","last_peered":"2024-09-17T08:21:26.775423+0000","last_clean":"2024-09-17T08:21:26.775423+0000","last_became_active":"2024-09-17T08:21:19.752680+0000","last_became_peered":"2024-09-17T08:21:19.752680+0000","last_unstale":"2024-09-17T08:21:26.775423+0000","last_undegraded":"2024-09-17T08:21:26.775423+0000","last_fullsized":"2024-09-17T08:21:26.775423+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:25.249461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777625+0000","last_change":"2024-09-17T08:21:23.768240+0000","last_active":"2024-09-17T08:21:26.777625+0000","last_peered":"2024-09-17T08:21:26.777625+0000","last_clean":"2024-09-17T08:21:26.777625+0000","last_became_active":"2024-09-17T08:21:23.767892+0000","last_became_peered":"2024-09-17T08:21:23.767892+0000","last_unstale":"2024-09-17T08:21:26.777625+0000","last_undegraded":"2024-09-17T08:21:26.777625+0000","last_fullsized":"2024-09-17T08:21:26.777625+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:00.201079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.778555+0000","last_change":"2024-09-17T08:21:25.779085+0000","last_active":"2024-09-17T08:21:26.778555+0000","last_peered":"2024-09-17T08:21:26.778555+0000","last_clean":"2024-09-17T08:21:26.778555+0000","last_became_active":"2024-09-17T08:21:25.778814+0000","last_became_peered":"2024-09-17T08:21:25.778814+0000","last_unstale":"2024-09-17T08:21:26.778555+0000","last_undegraded":"2024-09-17T08:21:26.778555+0000","last_fullsized":"2024-09-17T08:21:26.778555+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:30:54.657744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"30'12","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.927669+0000","last_change":"2024-09-17T08:21:21.760367+0000","last_active":"2024-09-17T08:21:26.927669+0000","last_peered":"2024-09-17T08:21:26.927669+0000","last_clean":"2024-09-17T08:21:26.927669+0000","last_became_active":"2024-09-17T08:21:21.759924+0000","last_became_peered":"2024-09-17T08:21:21.759924+0000","last_unstale":"2024-09-17T08:21:26.927669+0000","last_undegraded":"2024-09-17T08:21:26.927669+0000","last_fullsized":"2024-09-17T08:21:26.927669+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:50:31.120594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.779058+0000","last_change":"2024-09-17T08:21:19.755788+0000","last_active":"2024-09-17T08:21:26.779058+0000","last_peered":"2024-09-17T08:21:26.779058+0000","last_clean":"2024-09-17T08:21:26.779058+0000","last_became_active":"2024-09-17T08:21:19.755266+0000","last_became_peered":"2024-09-17T08:21:19.755266+0000","last_unstale":"2024-09-17T08:21:26.779058+0000","last_undegraded":"2024-09-17T08:21:26.779058+0000","last_fullsized":"2024-09-17T08:21:26.779058+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:09.532510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"30'8","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T08:21:52.900643+0000","last_change":"2024-09-17T08:21:23.770164+0000","last_active":"2024-09-17T08:21:52.900643+0000","last_peered":"2024-09-17T08:21:52.900643+0000","last_clean":"2024-09-17T08:21:52.900643+0000","last_became_active":"2024-09-17T08:21:23.769849+0000","last_became_peered":"2024-09-17T08:21:23.769849+0000","last_unstale":"2024-09-17T08:21:52.900643+0000","last_undegraded":"2024-09-17T08:21:52.900643+0000","last_fullsized":"2024-09-17T08:21:52.900643+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:40:11.343064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.954052+0000","last_change":"2024-09-17T08:21:25.779176+0000","last_active":"2024-09-17T08:21:26.954052+0000","last_peered":"2024-09-17T08:21:26.954052+0000","last_clean":"2024-09-17T08:21:26.954052+0000","last_became_active":"2024-09-17T08:21:25.778843+0000","last_became_peered":"2024-09-17T08:21:25.778843+0000","last_unstale":"2024-09-17T08:21:26.954052+0000","last_undegraded":"2024-09-17T08:21:26.954052+0000","last_fullsized":"2024-09-17T08:21:26.954052+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:32:51.078525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"30'12","reported_seq":34,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.045349+0000","last_change":"2024-09-17T08:21:21.760527+0000","last_active":"2024-09-17T08:21:27.045349+0000","last_peered":"2024-09-17T08:21:27.045349+0000","last_clean":"2024-09-17T08:21:27.045349+0000","last_became_active":"2024-09-17T08:21:21.760176+0000","last_became_peered":"2024-09-17T08:21:21.760176+0000","last_unstale":"2024-09-17T08:21:27.045349+0000","last_undegraded":"2024-09-17T08:21:27.045349+0000","last_fullsized":"2024-09-17T08:21:27.045349+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:15.048390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.780703+0000","last_change":"2024-09-17T08:21:19.753685+0000","last_active":"2024-09-17T08:21:26.780703+0000","last_peered":"2024-09-17T08:21:26.780703+0000","last_clean":"2024-09-17T08:21:26.780703+0000","last_became_active":"2024-09-17T08:21:19.753316+0000","last_became_peered":"2024-09-17T08:21:19.753316+0000","last_unstale":"2024-09-17T08:21:26.780703+0000","last_undegraded":"2024-09-17T08:21:26.780703+0000","last_fullsized":"2024-09-17T08:21:26.780703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:43.053033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776918+0000","last_change":"2024-09-17T08:21:23.764057+0000","last_active":"2024-09-17T08:21:26.776918+0000","last_peered":"2024-09-17T08:21:26.776918+0000","last_clean":"2024-09-17T08:21:26.776918+0000","last_became_active":"2024-09-17T08:21:23.763794+0000","last_became_peered":"2024-09-17T08:21:23.763794+0000","last_unstale":"2024-09-17T08:21:26.776918+0000","last_undegraded":"2024-09-17T08:21:26.776918+0000","last_fullsized":"2024-09-17T08:21:26.776918+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:18:15.786269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776013+0000","last_change":"2024-09-17T08:21:25.779140+0000","last_active":"2024-09-17T08:21:26.776013+0000","last_peered":"2024-09-17T08:21:26.776013+0000","last_clean":"2024-09-17T08:21:26.776013+0000","last_became_active":"2024-09-17T08:21:25.778645+0000","last_became_peered":"2024-09-17T08:21:25.778645+0000","last_unstale":"2024-09-17T08:21:26.776013+0000","last_undegraded":"2024-09-17T08:21:26.776013+0000","last_fullsized":"2024-09-17T08:21:26.776013+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:06.498952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"30'13","reported_seq":43,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.035148+0000","last_change":"2024-09-17T08:21:21.765897+0000","last_active":"2024-09-17T08:21:27.035148+0000","last_peered":"2024-09-17T08:21:27.035148+0000","last_clean":"2024-09-17T08:21:27.035148+0000","last_became_active":"2024-09-17T08:21:21.765433+0000","last_became_peered":"2024-09-17T08:21:21.765433+0000","last_unstale":"2024-09-17T08:21:27.035148+0000","last_undegraded":"2024-09-17T08:21:27.035148+0000","last_fullsized":"2024-09-17T08:21:27.035148+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:16:23.526808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777650+0000","last_change":"2024-09-17T08:21:19.754523+0000","last_active":"2024-09-17T08:21:26.777650+0000","last_peered":"2024-09-17T08:21:26.777650+0000","last_clean":"2024-09-17T08:21:26.777650+0000","last_became_active":"2024-09-17T08:21:19.754065+0000","last_became_peered":"2024-09-17T08:21:19.754065+0000","last_unstale":"2024-09-17T08:21:26.777650+0000","last_undegraded":"2024-09-17T08:21:26.777650+0000","last_fullsized":"2024-09-17T08:21:26.777650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:02.566708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"30'8","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T08:21:52.900454+0000","last_change":"2024-09-17T08:21:23.768550+0000","last_active":"2024-09-17T08:21:52.900454+0000","last_peered":"2024-09-17T08:21:52.900454+0000","last_clean":"2024-09-17T08:21:52.900454+0000","last_became_active":"2024-09-17T08:21:23.768234+0000","last_became_peered":"2024-09-17T08:21:23.768234+0000","last_unstale":"2024-09-17T08:21:52.900454+0000","last_undegraded":"2024-09-17T08:21:52.900454+0000","last_fullsized":"2024-09-17T08:21:52.900454+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:16.315747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953797+0000","last_change":"2024-09-17T08:21:25.779665+0000","last_active":"2024-09-17T08:21:26.953797+0000","last_peered":"2024-09-17T08:21:26.953797+0000","last_clean":"2024-09-17T08:21:26.953797+0000","last_became_active":"2024-09-17T08:21:25.779395+0000","last_became_peered":"2024-09-17T08:21:25.779395+0000","last_unstale":"2024-09-17T08:21:26.953797+0000","last_undegraded":"2024-09-17T08:21:26.953797+0000","last_fullsized":"2024-09-17T08:21:26.953797+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:02:17.500347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"30'28","reported_seq":72,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T08:21:52.900428+0000","last_change":"2024-09-17T08:21:21.761259+0000","last_active":"2024-09-17T08:21:52.900428+0000","last_peered":"2024-09-17T08:21:52.900428+0000","last_clean":"2024-09-17T08:21:52.900428+0000","last_became_active":"2024-09-17T08:21:21.760837+0000","last_became_peered":"2024-09-17T08:21:21.760837+0000","last_unstale":"2024-09-17T08:21:52.900428+0000","last_undegraded":"2024-09-17T08:21:52.900428+0000","last_fullsized":"2024-09-17T08:21:52.900428+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:09:23.512446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777594+0000","last_change":"2024-09-17T08:21:19.755691+0000","last_active":"2024-09-17T08:21:26.777594+0000","last_peered":"2024-09-17T08:21:26.777594+0000","last_clean":"2024-09-17T08:21:26.777594+0000","last_became_active":"2024-09-17T08:21:19.755459+0000","last_became_peered":"2024-09-17T08:21:19.755459+0000","last_unstale":"2024-09-17T08:21:26.777594+0000","last_undegraded":"2024-09-17T08:21:26.777594+0000","last_fullsized":"2024-09-17T08:21:26.777594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:53:59.366268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"30'8","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T08:21:52.900869+0000","last_change":"2024-09-17T08:21:23.767521+0000","last_active":"2024-09-17T08:21:52.900869+0000","last_peered":"2024-09-17T08:21:52.900869+0000","last_clean":"2024-09-17T08:21:52.900869+0000","last_became_active":"2024-09-17T08:21:23.767240+0000","last_became_peered":"2024-09-17T08:21:23.767240+0000","last_unstale":"2024-09-17T08:21:52.900869+0000","last_undegraded":"2024-09-17T08:21:52.900869+0000","last_fullsized":"2024-09-17T08:21:52.900869+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:55:50.735309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953850+0000","last_change":"2024-09-17T08:21:25.778372+0000","last_active":"2024-09-17T08:21:26.953850+0000","last_peered":"2024-09-17T08:21:26.953850+0000","last_clean":"2024-09-17T08:21:26.953850+0000","last_became_active":"2024-09-17T08:21:25.778033+0000","last_became_peered":"2024-09-17T08:21:25.778033+0000","last_unstale":"2024-09-17T08:21:26.953850+0000","last_undegraded":"2024-09-17T08:21:26.953850+0000","last_fullsized":"2024-09-17T08:21:26.953850+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:06:03.503230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"30'14","reported_seq":38,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.100761+0000","last_change":"2024-09-17T08:21:21.763296+0000","last_active":"2024-09-17T08:21:27.100761+0000","last_peered":"2024-09-17T08:21:27.100761+0000","last_clean":"2024-09-17T08:21:27.100761+0000","last_became_active":"2024-09-17T08:21:21.762228+0000","last_became_peered":"2024-09-17T08:21:21.762228+0000","last_unstale":"2024-09-17T08:21:27.100761+0000","last_undegraded":"2024-09-17T08:21:27.100761+0000","last_fullsized":"2024-09-17T08:21:27.100761+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:10.542907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776591+0000","last_change":"2024-09-17T08:21:19.753733+0000","last_active":"2024-09-17T08:21:26.776591+0000","last_peered":"2024-09-17T08:21:26.776591+0000","last_clean":"2024-09-17T08:21:26.776591+0000","last_became_active":"2024-09-17T08:21:19.753192+0000","last_became_peered":"2024-09-17T08:21:19.753192+0000","last_unstale":"2024-09-17T08:21:26.776591+0000","last_undegraded":"2024-09-17T08:21:26.776591+0000","last_fullsized":"2024-09-17T08:21:26.776591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:50.090763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777503+0000","last_change":"2024-09-17T08:21:23.766922+0000","last_active":"2024-09-17T08:21:26.777503+0000","last_peered":"2024-09-17T08:21:26.777503+0000","last_clean":"2024-09-17T08:21:26.777503+0000","last_became_active":"2024-09-17T08:21:23.766628+0000","last_became_peered":"2024-09-17T08:21:23.766628+0000","last_unstale":"2024-09-17T08:21:26.777503+0000","last_undegraded":"2024-09-17T08:21:26.777503+0000","last_fullsized":"2024-09-17T08:21:26.777503+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:56:03.480056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:29.452622+0000","last_change":"2024-09-17T08:21:25.780233+0000","last_active":"2024-09-17T08:21:29.452622+0000","last_peered":"2024-09-17T08:21:29.452622+0000","last_clean":"2024-09-17T08:21:29.452622+0000","last_became_active":"2024-09-17T08:21:25.779528+0000","last_became_peered":"2024-09-17T08:21:25.779528+0000","last_unstale":"2024-09-17T08:21:29.452622+0000","last_undegraded":"2024-09-17T08:21:29.452622+0000","last_fullsized":"2024-09-17T08:21:29.452622+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:41:26.599909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"30'19","reported_seq":52,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.087683+0000","last_change":"2024-09-17T08:21:21.760953+0000","last_active":"2024-09-17T08:21:27.087683+0000","last_peered":"2024-09-17T08:21:27.087683+0000","last_clean":"2024-09-17T08:21:27.087683+0000","last_became_active":"2024-09-17T08:21:21.760470+0000","last_became_peered":"2024-09-17T08:21:21.760470+0000","last_unstale":"2024-09-17T08:21:27.087683+0000","last_undegraded":"2024-09-17T08:21:27.087683+0000","last_fullsized":"2024-09-17T08:21:27.087683+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:06.666050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.779017+0000","last_change":"2024-09-17T08:21:19.753512+0000","last_active":"2024-09-17T08:21:26.779017+0000","last_peered":"2024-09-17T08:21:26.779017+0000","last_clean":"2024-09-17T08:21:26.779017+0000","last_became_active":"2024-09-17T08:21:19.752865+0000","last_became_peered":"2024-09-17T08:21:19.752865+0000","last_unstale":"2024-09-17T08:21:26.779017+0000","last_undegraded":"2024-09-17T08:21:26.779017+0000","last_fullsized":"2024-09-17T08:21:26.779017+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:50.469031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777088+0000","last_change":"2024-09-17T08:21:23.768280+0000","last_active":"2024-09-17T08:21:26.777088+0000","last_peered":"2024-09-17T08:21:26.777088+0000","last_clean":"2024-09-17T08:21:26.777088+0000","last_became_active":"2024-09-17T08:21:23.767885+0000","last_became_peered":"2024-09-17T08:21:23.767885+0000","last_unstale":"2024-09-17T08:21:26.777088+0000","last_undegraded":"2024-09-17T08:21:26.777088+0000","last_fullsized":"2024-09-17T08:21:26.777088+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:49.716822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:29.452776+0000","last_change":"2024-09-17T08:21:25.781133+0000","last_active":"2024-09-17T08:21:29.452776+0000","last_peered":"2024-09-17T08:21:29.452776+0000","last_clean":"2024-09-17T08:21:29.452776+0000","last_became_active":"2024-09-17T08:21:25.780757+0000","last_became_peered":"2024-09-17T08:21:25.780757+0000","last_unstale":"2024-09-17T08:21:29.452776+0000","last_undegraded":"2024-09-17T08:21:29.452776+0000","last_fullsized":"2024-09-17T08:21:29.452776+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:31.081090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"30'18","reported_seq":48,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.076522+0000","last_change":"2024-09-17T08:21:21.760535+0000","last_active":"2024-09-17T08:21:27.076522+0000","last_peered":"2024-09-17T08:21:27.076522+0000","last_clean":"2024-09-17T08:21:27.076522+0000","last_became_active":"2024-09-17T08:21:21.760237+0000","last_became_peered":"2024-09-17T08:21:21.760237+0000","last_unstale":"2024-09-17T08:21:27.076522+0000","last_undegraded":"2024-09-17T08:21:27.076522+0000","last_fullsized":"2024-09-17T08:21:27.076522+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:34:19.786977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.775716+0000","last_change":"2024-09-17T08:21:19.754333+0000","last_active":"2024-09-17T08:21:26.775716+0000","last_peered":"2024-09-17T08:21:26.775716+0000","last_clean":"2024-09-17T08:21:26.775716+0000","last_became_active":"2024-09-17T08:21:19.753294+0000","last_became_peered":"2024-09-17T08:21:19.753294+0000","last_unstale":"2024-09-17T08:21:26.775716+0000","last_undegraded":"2024-09-17T08:21:26.775716+0000","last_fullsized":"2024-09-17T08:21:26.775716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:49:01.893917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777288+0000","last_change":"2024-09-17T08:21:23.767277+0000","last_active":"2024-09-17T08:21:26.777288+0000","last_peered":"2024-09-17T08:21:26.777288+0000","last_clean":"2024-09-17T08:21:26.777288+0000","last_became_active":"2024-09-17T08:21:23.766902+0000","last_became_peered":"2024-09-17T08:21:23.766902+0000","last_unstale":"2024-09-17T08:21:26.777288+0000","last_undegraded":"2024-09-17T08:21:26.777288+0000","last_fullsized":"2024-09-17T08:21:26.777288+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:44:51.770796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:29.452826+0000","last_change":"2024-09-17T08:21:25.780230+0000","last_active":"2024-09-17T08:21:29.452826+0000","last_peered":"2024-09-17T08:21:29.452826+0000","last_clean":"2024-09-17T08:21:29.452826+0000","last_became_active":"2024-09-17T08:21:25.779909+0000","last_became_peered":"2024-09-17T08:21:25.779909+0000","last_unstale":"2024-09-17T08:21:29.452826+0000","last_undegraded":"2024-09-17T08:21:29.452826+0000","last_fullsized":"2024-09-17T08:21:29.452826+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:09:26.699991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"30'14","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.109828+0000","last_change":"2024-09-17T08:21:21.761155+0000","last_active":"2024-09-17T08:21:27.109828+0000","last_peered":"2024-09-17T08:21:27.109828+0000","last_clean":"2024-09-17T08:21:27.109828+0000","last_became_active":"2024-09-17T08:21:21.760070+0000","last_became_peered":"2024-09-17T08:21:21.760070+0000","last_unstale":"2024-09-17T08:21:27.109828+0000","last_undegraded":"2024-09-17T08:21:27.109828+0000","last_fullsized":"2024-09-17T08:21:27.109828+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:25.057234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.778960+0000","last_change":"2024-09-17T08:21:19.753672+0000","last_active":"2024-09-17T08:21:26.778960+0000","last_peered":"2024-09-17T08:21:26.778960+0000","last_clean":"2024-09-17T08:21:26.778960+0000","last_became_active":"2024-09-17T08:21:19.753187+0000","last_became_peered":"2024-09-17T08:21:19.753187+0000","last_unstale":"2024-09-17T08:21:26.778960+0000","last_undegraded":"2024-09-17T08:21:26.778960+0000","last_fullsized":"2024-09-17T08:21:26.778960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:55.086952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777481+0000","last_change":"2024-09-17T08:21:23.764097+0000","last_active":"2024-09-17T08:21:26.777481+0000","last_peered":"2024-09-17T08:21:26.777481+0000","last_clean":"2024-09-17T08:21:26.777481+0000","last_became_active":"2024-09-17T08:21:23.763769+0000","last_became_peered":"2024-09-17T08:21:23.763769+0000","last_unstale":"2024-09-17T08:21:26.777481+0000","last_undegraded":"2024-09-17T08:21:26.777481+0000","last_fullsized":"2024-09-17T08:21:26.777481+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:22:28.012538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.954083+0000","last_change":"2024-09-17T08:21:25.777566+0000","last_active":"2024-09-17T08:21:26.954083+0000","last_peered":"2024-09-17T08:21:26.954083+0000","last_clean":"2024-09-17T08:21:26.954083+0000","last_became_active":"2024-09-17T08:21:25.777255+0000","last_became_peered":"2024-09-17T08:21:25.777255+0000","last_unstale":"2024-09-17T08:21:26.954083+0000","last_undegraded":"2024-09-17T08:21:26.954083+0000","last_fullsized":"2024-09-17T08:21:26.954083+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:26:37.643645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"30'10","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.095275+0000","last_change":"2024-09-17T08:21:21.760931+0000","last_active":"2024-09-17T08:21:27.095275+0000","last_peered":"2024-09-17T08:21:27.095275+0000","last_clean":"2024-09-17T08:21:27.095275+0000","last_became_active":"2024-09-17T08:21:21.760489+0000","last_became_peered":"2024-09-17T08:21:21.760489+0000","last_unstale":"2024-09-17T08:21:27.095275+0000","last_undegraded":"2024-09-17T08:21:27.095275+0000","last_fullsized":"2024-09-17T08:21:27.095275+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:34.786006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.779553+0000","last_change":"2024-09-17T08:21:19.753402+0000","last_active":"2024-09-17T08:21:26.779553+0000","last_peered":"2024-09-17T08:21:26.779553+0000","last_clean":"2024-09-17T08:21:26.779553+0000","last_became_active":"2024-09-17T08:21:19.753000+0000","last_became_peered":"2024-09-17T08:21:19.753000+0000","last_unstale":"2024-09-17T08:21:26.779553+0000","last_undegraded":"2024-09-17T08:21:26.779553+0000","last_fullsized":"2024-09-17T08:21:26.779553+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:55:13.233754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"31'208","reported_seq":302,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T08:21:47.194383+0000","last_change":"2024-09-17T08:21:07.119870+0000","last_active":"2024-09-17T08:21:47.194383+0000","last_peered":"2024-09-17T08:21:47.194383+0000","last_clean":"2024-09-17T08:21:47.194383+0000","last_became_active":"2024-09-17T08:21:07.118876+0000","last_became_peered":"2024-09-17T08:21:07.118876+0000","last_unstale":"2024-09-17T08:21:47.194383+0000","last_undegraded":"2024-09-17T08:21:47.194383+0000","last_fullsized":"2024-09-17T08:21:47.194383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:06.110207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:06.110207+0000","last_clean_scrub_stamp":"2024-09-17T08:21:06.110207+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:21.814379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953989+0000","last_change":"2024-09-17T08:21:23.768358+0000","last_active":"2024-09-17T08:21:26.953989+0000","last_peered":"2024-09-17T08:21:26.953989+0000","last_clean":"2024-09-17T08:21:26.953989+0000","last_became_active":"2024-09-17T08:21:23.767553+0000","last_became_peered":"2024-09-17T08:21:23.767553+0000","last_unstale":"2024-09-17T08:21:26.953989+0000","last_undegraded":"2024-09-17T08:21:26.953989+0000","last_fullsized":"2024-09-17T08:21:26.953989+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:34.815653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777755+0000","last_change":"2024-09-17T08:21:25.779201+0000","last_active":"2024-09-17T08:21:26.777755+0000","last_peered":"2024-09-17T08:21:26.777755+0000","last_clean":"2024-09-17T08:21:26.777755+0000","last_became_active":"2024-09-17T08:21:25.778435+0000","last_became_peered":"2024-09-17T08:21:25.778435+0000","last_unstale":"2024-09-17T08:21:26.777755+0000","last_undegraded":"2024-09-17T08:21:26.777755+0000","last_fullsized":"2024-09-17T08:21:26.777755+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:55:19.814835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'17","reported_seq":44,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.102894+0000","last_change":"2024-09-17T08:21:21.760682+0000","last_active":"2024-09-17T08:21:27.102894+0000","last_peered":"2024-09-17T08:21:27.102894+0000","last_clean":"2024-09-17T08:21:27.102894+0000","last_became_active":"2024-09-17T08:21:21.760171+0000","last_became_peered":"2024-09-17T08:21:21.760171+0000","last_unstale":"2024-09-17T08:21:27.102894+0000","last_undegraded":"2024-09-17T08:21:27.102894+0000","last_fullsized":"2024-09-17T08:21:27.102894+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:44.217636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776861+0000","last_change":"2024-09-17T08:21:19.753083+0000","last_active":"2024-09-17T08:21:26.776861+0000","last_peered":"2024-09-17T08:21:26.776861+0000","last_clean":"2024-09-17T08:21:26.776861+0000","last_became_active":"2024-09-17T08:21:19.752793+0000","last_became_peered":"2024-09-17T08:21:19.752793+0000","last_unstale":"2024-09-17T08:21:26.776861+0000","last_undegraded":"2024-09-17T08:21:26.776861+0000","last_fullsized":"2024-09-17T08:21:26.776861+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:41:31.625936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953840+0000","last_change":"2024-09-17T08:21:23.768588+0000","last_active":"2024-09-17T08:21:26.953840+0000","last_peered":"2024-09-17T08:21:26.953840+0000","last_clean":"2024-09-17T08:21:26.953840+0000","last_became_active":"2024-09-17T08:21:23.768078+0000","last_became_peered":"2024-09-17T08:21:23.768078+0000","last_unstale":"2024-09-17T08:21:26.953840+0000","last_undegraded":"2024-09-17T08:21:26.953840+0000","last_fullsized":"2024-09-17T08:21:26.953840+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:38.348121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777549+0000","last_change":"2024-09-17T08:21:25.779025+0000","last_active":"2024-09-17T08:21:26.777549+0000","last_peered":"2024-09-17T08:21:26.777549+0000","last_clean":"2024-09-17T08:21:26.777549+0000","last_became_active":"2024-09-17T08:21:25.778664+0000","last_became_peered":"2024-09-17T08:21:25.778664+0000","last_unstale":"2024-09-17T08:21:26.777549+0000","last_undegraded":"2024-09-17T08:21:26.777549+0000","last_fullsized":"2024-09-17T08:21:26.777549+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:31:57.823770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"30'10","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.056688+0000","last_change":"2024-09-17T08:21:21.760419+0000","last_active":"2024-09-17T08:21:27.056688+0000","last_peered":"2024-09-17T08:21:27.056688+0000","last_clean":"2024-09-17T08:21:27.056688+0000","last_became_active":"2024-09-17T08:21:21.760102+0000","last_became_peered":"2024-09-17T08:21:21.760102+0000","last_unstale":"2024-09-17T08:21:27.056688+0000","last_undegraded":"2024-09-17T08:21:27.056688+0000","last_fullsized":"2024-09-17T08:21:27.056688+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:49.052441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777303+0000","last_change":"2024-09-17T08:21:19.755785+0000","last_active":"2024-09-17T08:21:26.777303+0000","last_peered":"2024-09-17T08:21:26.777303+0000","last_clean":"2024-09-17T08:21:26.777303+0000","last_became_active":"2024-09-17T08:21:19.755490+0000","last_became_peered":"2024-09-17T08:21:19.755490+0000","last_unstale":"2024-09-17T08:21:26.777303+0000","last_undegraded":"2024-09-17T08:21:26.777303+0000","last_fullsized":"2024-09-17T08:21:26.777303+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:59.056786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776568+0000","last_change":"2024-09-17T08:21:23.767056+0000","last_active":"2024-09-17T08:21:26.776568+0000","last_peered":"2024-09-17T08:21:26.776568+0000","last_clean":"2024-09-17T08:21:26.776568+0000","last_became_active":"2024-09-17T08:21:23.766760+0000","last_became_peered":"2024-09-17T08:21:23.766760+0000","last_unstale":"2024-09-17T08:21:26.776568+0000","last_undegraded":"2024-09-17T08:21:26.776568+0000","last_fullsized":"2024-09-17T08:21:26.776568+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:43.136151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:29.452565+0000","last_change":"2024-09-17T08:21:25.774852+0000","last_active":"2024-09-17T08:21:29.452565+0000","last_peered":"2024-09-17T08:21:29.452565+0000","last_clean":"2024-09-17T08:21:29.452565+0000","last_became_active":"2024-09-17T08:21:25.774558+0000","last_became_peered":"2024-09-17T08:21:25.774558+0000","last_unstale":"2024-09-17T08:21:29.452565+0000","last_undegraded":"2024-09-17T08:21:29.452565+0000","last_fullsized":"2024-09-17T08:21:29.452565+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:36.454496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"30'15","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.127501+0000","last_change":"2024-09-17T08:21:21.763292+0000","last_active":"2024-09-17T08:21:27.127501+0000","last_peered":"2024-09-17T08:21:27.127501+0000","last_clean":"2024-09-17T08:21:27.127501+0000","last_became_active":"2024-09-17T08:21:21.762260+0000","last_became_peered":"2024-09-17T08:21:21.762260+0000","last_unstale":"2024-09-17T08:21:27.127501+0000","last_undegraded":"2024-09-17T08:21:27.127501+0000","last_fullsized":"2024-09-17T08:21:27.127501+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:14.413275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777464+0000","last_change":"2024-09-17T08:21:19.754334+0000","last_active":"2024-09-17T08:21:26.777464+0000","last_peered":"2024-09-17T08:21:26.777464+0000","last_clean":"2024-09-17T08:21:26.777464+0000","last_became_active":"2024-09-17T08:21:19.753029+0000","last_became_peered":"2024-09-17T08:21:19.753029+0000","last_unstale":"2024-09-17T08:21:26.777464+0000","last_undegraded":"2024-09-17T08:21:26.777464+0000","last_fullsized":"2024-09-17T08:21:26.777464+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:16:19.205841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777100+0000","last_change":"2024-09-17T08:21:23.768393+0000","last_active":"2024-09-17T08:21:26.777100+0000","last_peered":"2024-09-17T08:21:26.777100+0000","last_clean":"2024-09-17T08:21:26.777100+0000","last_became_active":"2024-09-17T08:21:23.767769+0000","last_became_peered":"2024-09-17T08:21:23.767769+0000","last_unstale":"2024-09-17T08:21:26.777100+0000","last_undegraded":"2024-09-17T08:21:26.777100+0000","last_fullsized":"2024-09-17T08:21:26.777100+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:15.462728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953777+0000","last_change":"2024-09-17T08:21:25.778567+0000","last_active":"2024-09-17T08:21:26.953777+0000","last_peered":"2024-09-17T08:21:26.953777+0000","last_clean":"2024-09-17T08:21:26.953777+0000","last_became_active":"2024-09-17T08:21:25.777796+0000","last_became_peered":"2024-09-17T08:21:25.777796+0000","last_unstale":"2024-09-17T08:21:26.953777+0000","last_undegraded":"2024-09-17T08:21:26.953777+0000","last_fullsized":"2024-09-17T08:21:26.953777+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:28.766660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"30'11","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.104980+0000","last_change":"2024-09-17T08:21:21.760425+0000","last_active":"2024-09-17T08:21:27.104980+0000","last_peered":"2024-09-17T08:21:27.104980+0000","last_clean":"2024-09-17T08:21:27.104980+0000","last_became_active":"2024-09-17T08:21:21.760131+0000","last_became_peered":"2024-09-17T08:21:21.760131+0000","last_unstale":"2024-09-17T08:21:27.104980+0000","last_undegraded":"2024-09-17T08:21:27.104980+0000","last_fullsized":"2024-09-17T08:21:27.104980+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:43:06.079541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"23'2","reported_seq":25,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.886368+0000","last_change":"2024-09-17T08:21:19.753602+0000","last_active":"2024-09-17T08:21:26.886368+0000","last_peered":"2024-09-17T08:21:26.886368+0000","last_clean":"2024-09-17T08:21:26.886368+0000","last_became_active":"2024-09-17T08:21:19.753117+0000","last_became_peered":"2024-09-17T08:21:19.753117+0000","last_unstale":"2024-09-17T08:21:26.886368+0000","last_undegraded":"2024-09-17T08:21:26.886368+0000","last_fullsized":"2024-09-17T08:21:26.886368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:56:05.089959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"30'8","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T08:21:52.900899+0000","last_change":"2024-09-17T08:21:23.763581+0000","last_active":"2024-09-17T08:21:52.900899+0000","last_peered":"2024-09-17T08:21:52.900899+0000","last_clean":"2024-09-17T08:21:52.900899+0000","last_became_active":"2024-09-17T08:21:23.763304+0000","last_became_peered":"2024-09-17T08:21:23.763304+0000","last_unstale":"2024-09-17T08:21:52.900899+0000","last_undegraded":"2024-09-17T08:21:52.900899+0000","last_fullsized":"2024-09-17T08:21:52.900899+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:04.870366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.775999+0000","last_change":"2024-09-17T08:21:25.778390+0000","last_active":"2024-09-17T08:21:26.775999+0000","last_peered":"2024-09-17T08:21:26.775999+0000","last_clean":"2024-09-17T08:21:26.775999+0000","last_became_active":"2024-09-17T08:21:25.777933+0000","last_became_peered":"2024-09-17T08:21:25.777933+0000","last_unstale":"2024-09-17T08:21:26.775999+0000","last_undegraded":"2024-09-17T08:21:26.775999+0000","last_fullsized":"2024-09-17T08:21:26.775999+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:31.568262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"30'11","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.112348+0000","last_change":"2024-09-17T08:21:21.760651+0000","last_active":"2024-09-17T08:21:27.112348+0000","last_peered":"2024-09-17T08:21:27.112348+0000","last_clean":"2024-09-17T08:21:27.112348+0000","last_became_active":"2024-09-17T08:21:21.760336+0000","last_became_peered":"2024-09-17T08:21:21.760336+0000","last_unstale":"2024-09-17T08:21:27.112348+0000","last_undegraded":"2024-09-17T08:21:27.112348+0000","last_fullsized":"2024-09-17T08:21:27.112348+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:58.553273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776209+0000","last_change":"2024-09-17T08:21:19.753200+0000","last_active":"2024-09-17T08:21:26.776209+0000","last_peered":"2024-09-17T08:21:26.776209+0000","last_clean":"2024-09-17T08:21:26.776209+0000","last_became_active":"2024-09-17T08:21:19.752610+0000","last_became_peered":"2024-09-17T08:21:19.752610+0000","last_unstale":"2024-09-17T08:21:26.776209+0000","last_undegraded":"2024-09-17T08:21:26.776209+0000","last_fullsized":"2024-09-17T08:21:26.776209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:04:38.663894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777757+0000","last_change":"2024-09-17T08:21:23.768110+0000","last_active":"2024-09-17T08:21:26.777757+0000","last_peered":"2024-09-17T08:21:26.777757+0000","last_clean":"2024-09-17T08:21:26.777757+0000","last_became_active":"2024-09-17T08:21:23.767776+0000","last_became_peered":"2024-09-17T08:21:23.767776+0000","last_unstale":"2024-09-17T08:21:26.777757+0000","last_undegraded":"2024-09-17T08:21:26.777757+0000","last_fullsized":"2024-09-17T08:21:26.777757+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:55.194976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:29.452969+0000","last_change":"2024-09-17T08:21:25.781994+0000","last_active":"2024-09-17T08:21:29.452969+0000","last_peered":"2024-09-17T08:21:29.452969+0000","last_clean":"2024-09-17T08:21:29.452969+0000","last_became_active":"2024-09-17T08:21:25.781704+0000","last_became_peered":"2024-09-17T08:21:25.781704+0000","last_unstale":"2024-09-17T08:21:29.452969+0000","last_undegraded":"2024-09-17T08:21:29.452969+0000","last_fullsized":"2024-09-17T08:21:29.452969+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:09.488605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"30'4","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.931210+0000","last_change":"2024-09-17T08:21:21.760993+0000","last_active":"2024-09-17T08:21:26.931210+0000","last_peered":"2024-09-17T08:21:26.931210+0000","last_clean":"2024-09-17T08:21:26.931210+0000","last_became_active":"2024-09-17T08:21:21.760529+0000","last_became_peered":"2024-09-17T08:21:21.760529+0000","last_unstale":"2024-09-17T08:21:26.931210+0000","last_undegraded":"2024-09-17T08:21:26.931210+0000","last_fullsized":"2024-09-17T08:21:26.931210+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:25.529349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.778825+0000","last_change":"2024-09-17T08:21:19.754084+0000","last_active":"2024-09-17T08:21:26.778825+0000","last_peered":"2024-09-17T08:21:26.778825+0000","last_clean":"2024-09-17T08:21:26.778825+0000","last_became_active":"2024-09-17T08:21:19.753350+0000","last_became_peered":"2024-09-17T08:21:19.753350+0000","last_unstale":"2024-09-17T08:21:26.778825+0000","last_undegraded":"2024-09-17T08:21:26.778825+0000","last_fullsized":"2024-09-17T08:21:26.778825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:30:21.712101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777678+0000","last_change":"2024-09-17T08:21:23.767933+0000","last_active":"2024-09-17T08:21:26.777678+0000","last_peered":"2024-09-17T08:21:26.777678+0000","last_clean":"2024-09-17T08:21:26.777678+0000","last_became_active":"2024-09-17T08:21:23.767478+0000","last_became_peered":"2024-09-17T08:21:23.767478+0000","last_unstale":"2024-09-17T08:21:26.777678+0000","last_undegraded":"2024-09-17T08:21:26.777678+0000","last_fullsized":"2024-09-17T08:21:26.777678+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:00:42.403121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.954034+0000","last_change":"2024-09-17T08:21:25.778517+0000","last_active":"2024-09-17T08:21:26.954034+0000","last_peered":"2024-09-17T08:21:26.954034+0000","last_clean":"2024-09-17T08:21:26.954034+0000","last_became_active":"2024-09-17T08:21:25.778044+0000","last_became_peered":"2024-09-17T08:21:25.778044+0000","last_unstale":"2024-09-17T08:21:26.954034+0000","last_undegraded":"2024-09-17T08:21:26.954034+0000","last_fullsized":"2024-09-17T08:21:26.954034+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:54.886050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"30'11","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.080636+0000","last_change":"2024-09-17T08:21:21.760413+0000","last_active":"2024-09-17T08:21:27.080636+0000","last_peered":"2024-09-17T08:21:27.080636+0000","last_clean":"2024-09-17T08:21:27.080636+0000","last_became_active":"2024-09-17T08:21:21.760077+0000","last_became_peered":"2024-09-17T08:21:21.760077+0000","last_unstale":"2024-09-17T08:21:27.080636+0000","last_undegraded":"2024-09-17T08:21:27.080636+0000","last_fullsized":"2024-09-17T08:21:27.080636+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:17:06.909689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776528+0000","last_change":"2024-09-17T08:21:19.753028+0000","last_active":"2024-09-17T08:21:26.776528+0000","last_peered":"2024-09-17T08:21:26.776528+0000","last_clean":"2024-09-17T08:21:26.776528+0000","last_became_active":"2024-09-17T08:21:19.752711+0000","last_became_peered":"2024-09-17T08:21:19.752711+0000","last_unstale":"2024-09-17T08:21:26.776528+0000","last_undegraded":"2024-09-17T08:21:26.776528+0000","last_fullsized":"2024-09-17T08:21:26.776528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:14:49.844405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"30'8","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T08:21:52.900880+0000","last_change":"2024-09-17T08:21:23.767878+0000","last_active":"2024-09-17T08:21:52.900880+0000","last_peered":"2024-09-17T08:21:52.900880+0000","last_clean":"2024-09-17T08:21:52.900880+0000","last_became_active":"2024-09-17T08:21:23.767597+0000","last_became_peered":"2024-09-17T08:21:23.767597+0000","last_unstale":"2024-09-17T08:21:52.900880+0000","last_undegraded":"2024-09-17T08:21:52.900880+0000","last_fullsized":"2024-09-17T08:21:52.900880+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:36.067005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953999+0000","last_change":"2024-09-17T08:21:25.778812+0000","last_active":"2024-09-17T08:21:26.953999+0000","last_peered":"2024-09-17T08:21:26.953999+0000","last_clean":"2024-09-17T08:21:26.953999+0000","last_became_active":"2024-09-17T08:21:25.778288+0000","last_became_peered":"2024-09-17T08:21:25.778288+0000","last_unstale":"2024-09-17T08:21:26.953999+0000","last_undegraded":"2024-09-17T08:21:26.953999+0000","last_fullsized":"2024-09-17T08:21:26.953999+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:03:38.851264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.008513+0000","last_change":"2024-09-17T08:21:21.760586+0000","last_active":"2024-09-17T08:21:27.008513+0000","last_peered":"2024-09-17T08:21:27.008513+0000","last_clean":"2024-09-17T08:21:27.008513+0000","last_became_active":"2024-09-17T08:21:21.760206+0000","last_became_peered":"2024-09-17T08:21:21.760206+0000","last_unstale":"2024-09-17T08:21:27.008513+0000","last_undegraded":"2024-09-17T08:21:27.008513+0000","last_fullsized":"2024-09-17T08:21:27.008513+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:46:00.593846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776787+0000","last_change":"2024-09-17T08:21:19.753810+0000","last_active":"2024-09-17T08:21:26.776787+0000","last_peered":"2024-09-17T08:21:26.776787+0000","last_clean":"2024-09-17T08:21:26.776787+0000","last_became_active":"2024-09-17T08:21:19.753338+0000","last_became_peered":"2024-09-17T08:21:19.753338+0000","last_unstale":"2024-09-17T08:21:26.776787+0000","last_undegraded":"2024-09-17T08:21:26.776787+0000","last_fullsized":"2024-09-17T08:21:26.776787+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:46:49.961395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"30'8","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T08:21:52.900853+0000","last_change":"2024-09-17T08:21:23.764798+0000","last_active":"2024-09-17T08:21:52.900853+0000","last_peered":"2024-09-17T08:21:52.900853+0000","last_clean":"2024-09-17T08:21:52.900853+0000","last_became_active":"2024-09-17T08:21:23.764557+0000","last_became_peered":"2024-09-17T08:21:23.764557+0000","last_unstale":"2024-09-17T08:21:52.900853+0000","last_undegraded":"2024-09-17T08:21:52.900853+0000","last_fullsized":"2024-09-17T08:21:52.900853+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:04:13.207460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:29.452683+0000","last_change":"2024-09-17T08:21:25.780002+0000","last_active":"2024-09-17T08:21:29.452683+0000","last_peered":"2024-09-17T08:21:29.452683+0000","last_clean":"2024-09-17T08:21:29.452683+0000","last_became_active":"2024-09-17T08:21:25.779697+0000","last_became_peered":"2024-09-17T08:21:25.779697+0000","last_unstale":"2024-09-17T08:21:29.452683+0000","last_undegraded":"2024-09-17T08:21:29.452683+0000","last_fullsized":"2024-09-17T08:21:29.452683+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:46.975857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.092201+0000","last_change":"2024-09-17T08:21:21.760661+0000","last_active":"2024-09-17T08:21:27.092201+0000","last_peered":"2024-09-17T08:21:27.092201+0000","last_clean":"2024-09-17T08:21:27.092201+0000","last_became_active":"2024-09-17T08:21:21.759994+0000","last_became_peered":"2024-09-17T08:21:21.759994+0000","last_unstale":"2024-09-17T08:21:27.092201+0000","last_undegraded":"2024-09-17T08:21:27.092201+0000","last_fullsized":"2024-09-17T08:21:27.092201+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:34.464810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.780483+0000","last_change":"2024-09-17T08:21:19.753492+0000","last_active":"2024-09-17T08:21:26.780483+0000","last_peered":"2024-09-17T08:21:26.780483+0000","last_clean":"2024-09-17T08:21:26.780483+0000","last_became_active":"2024-09-17T08:21:19.753104+0000","last_became_peered":"2024-09-17T08:21:19.753104+0000","last_unstale":"2024-09-17T08:21:26.780483+0000","last_undegraded":"2024-09-17T08:21:26.780483+0000","last_fullsized":"2024-09-17T08:21:26.780483+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:48.506654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776214+0000","last_change":"2024-09-17T08:21:23.766953+0000","last_active":"2024-09-17T08:21:26.776214+0000","last_peered":"2024-09-17T08:21:26.776214+0000","last_clean":"2024-09-17T08:21:26.776214+0000","last_became_active":"2024-09-17T08:21:23.766357+0000","last_became_peered":"2024-09-17T08:21:23.766357+0000","last_unstale":"2024-09-17T08:21:26.776214+0000","last_undegraded":"2024-09-17T08:21:26.776214+0000","last_fullsized":"2024-09-17T08:21:26.776214+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:15:03.591049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776640+0000","last_change":"2024-09-17T08:21:25.778423+0000","last_active":"2024-09-17T08:21:26.776640+0000","last_peered":"2024-09-17T08:21:26.776640+0000","last_clean":"2024-09-17T08:21:26.776640+0000","last_became_active":"2024-09-17T08:21:25.777964+0000","last_became_peered":"2024-09-17T08:21:25.777964+0000","last_unstale":"2024-09-17T08:21:26.776640+0000","last_undegraded":"2024-09-17T08:21:26.776640+0000","last_fullsized":"2024-09-17T08:21:26.776640+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:35.618652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"30'10","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.089837+0000","last_change":"2024-09-17T08:21:21.761279+0000","last_active":"2024-09-17T08:21:27.089837+0000","last_peered":"2024-09-17T08:21:27.089837+0000","last_clean":"2024-09-17T08:21:27.089837+0000","last_became_active":"2024-09-17T08:21:21.760825+0000","last_became_peered":"2024-09-17T08:21:21.760825+0000","last_unstale":"2024-09-17T08:21:27.089837+0000","last_undegraded":"2024-09-17T08:21:27.089837+0000","last_fullsized":"2024-09-17T08:21:27.089837+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:18:14.553324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"23'1","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.886712+0000","last_change":"2024-09-17T08:21:19.755088+0000","last_active":"2024-09-17T08:21:26.886712+0000","last_peered":"2024-09-17T08:21:26.886712+0000","last_clean":"2024-09-17T08:21:26.886712+0000","last_became_active":"2024-09-17T08:21:19.754831+0000","last_became_peered":"2024-09-17T08:21:19.754831+0000","last_unstale":"2024-09-17T08:21:26.886712+0000","last_undegraded":"2024-09-17T08:21:26.886712+0000","last_fullsized":"2024-09-17T08:21:26.886712+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:00:46.174385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953904+0000","last_change":"2024-09-17T08:21:23.771371+0000","last_active":"2024-09-17T08:21:26.953904+0000","last_peered":"2024-09-17T08:21:26.953904+0000","last_clean":"2024-09-17T08:21:26.953904+0000","last_became_active":"2024-09-17T08:21:23.771076+0000","last_became_peered":"2024-09-17T08:21:23.771076+0000","last_unstale":"2024-09-17T08:21:26.953904+0000","last_undegraded":"2024-09-17T08:21:26.953904+0000","last_fullsized":"2024-09-17T08:21:26.953904+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:46.042493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777979+0000","last_change":"2024-09-17T08:21:25.779255+0000","last_active":"2024-09-17T08:21:26.777979+0000","last_peered":"2024-09-17T08:21:26.777979+0000","last_clean":"2024-09-17T08:21:26.777979+0000","last_became_active":"2024-09-17T08:21:25.778791+0000","last_became_peered":"2024-09-17T08:21:25.778791+0000","last_unstale":"2024-09-17T08:21:26.777979+0000","last_undegraded":"2024-09-17T08:21:26.777979+0000","last_fullsized":"2024-09-17T08:21:26.777979+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:13.928809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"30'6","reported_seq":25,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.071447+0000","last_change":"2024-09-17T08:21:21.765324+0000","last_active":"2024-09-17T08:21:27.071447+0000","last_peered":"2024-09-17T08:21:27.071447+0000","last_clean":"2024-09-17T08:21:27.071447+0000","last_became_active":"2024-09-17T08:21:21.764774+0000","last_became_peered":"2024-09-17T08:21:21.764774+0000","last_unstale":"2024-09-17T08:21:27.071447+0000","last_undegraded":"2024-09-17T08:21:27.071447+0000","last_fullsized":"2024-09-17T08:21:27.071447+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:06.870820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.780359+0000","last_change":"2024-09-17T08:21:19.757862+0000","last_active":"2024-09-17T08:21:26.780359+0000","last_peered":"2024-09-17T08:21:26.780359+0000","last_clean":"2024-09-17T08:21:26.780359+0000","last_became_active":"2024-09-17T08:21:19.757553+0000","last_became_peered":"2024-09-17T08:21:19.757553+0000","last_unstale":"2024-09-17T08:21:26.780359+0000","last_undegraded":"2024-09-17T08:21:26.780359+0000","last_fullsized":"2024-09-17T08:21:26.780359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:46:02.143139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.775752+0000","last_change":"2024-09-17T08:21:23.768657+0000","last_active":"2024-09-17T08:21:26.775752+0000","last_peered":"2024-09-17T08:21:26.775752+0000","last_clean":"2024-09-17T08:21:26.775752+0000","last_became_active":"2024-09-17T08:21:23.768357+0000","last_became_peered":"2024-09-17T08:21:23.768357+0000","last_unstale":"2024-09-17T08:21:26.775752+0000","last_undegraded":"2024-09-17T08:21:26.775752+0000","last_fullsized":"2024-09-17T08:21:26.775752+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:27:14.670258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953768+0000","last_change":"2024-09-17T08:21:25.778156+0000","last_active":"2024-09-17T08:21:26.953768+0000","last_peered":"2024-09-17T08:21:26.953768+0000","last_clean":"2024-09-17T08:21:26.953768+0000","last_became_active":"2024-09-17T08:21:25.777699+0000","last_became_peered":"2024-09-17T08:21:25.777699+0000","last_unstale":"2024-09-17T08:21:26.953768+0000","last_undegraded":"2024-09-17T08:21:26.953768+0000","last_fullsized":"2024-09-17T08:21:26.953768+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:46:57.612816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.944689+0000","last_change":"2024-09-17T08:21:21.763281+0000","last_active":"2024-09-17T08:21:26.944689+0000","last_peered":"2024-09-17T08:21:26.944689+0000","last_clean":"2024-09-17T08:21:26.944689+0000","last_became_active":"2024-09-17T08:21:21.762313+0000","last_became_peered":"2024-09-17T08:21:21.762313+0000","last_unstale":"2024-09-17T08:21:26.944689+0000","last_undegraded":"2024-09-17T08:21:26.944689+0000","last_fullsized":"2024-09-17T08:21:26.944689+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:28:52.720354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.780859+0000","last_change":"2024-09-17T08:21:19.754848+0000","last_active":"2024-09-17T08:21:26.780859+0000","last_peered":"2024-09-17T08:21:26.780859+0000","last_clean":"2024-09-17T08:21:26.780859+0000","last_became_active":"2024-09-17T08:21:19.754615+0000","last_became_peered":"2024-09-17T08:21:19.754615+0000","last_unstale":"2024-09-17T08:21:26.780859+0000","last_undegraded":"2024-09-17T08:21:26.780859+0000","last_fullsized":"2024-09-17T08:21:26.780859+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:38:15.073793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777340+0000","last_change":"2024-09-17T08:21:23.766760+0000","last_active":"2024-09-17T08:21:26.777340+0000","last_peered":"2024-09-17T08:21:26.777340+0000","last_clean":"2024-09-17T08:21:26.777340+0000","last_became_active":"2024-09-17T08:21:23.766367+0000","last_became_peered":"2024-09-17T08:21:23.766367+0000","last_unstale":"2024-09-17T08:21:26.777340+0000","last_undegraded":"2024-09-17T08:21:26.777340+0000","last_fullsized":"2024-09-17T08:21:26.777340+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:32.505627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953723+0000","last_change":"2024-09-17T08:21:25.778727+0000","last_active":"2024-09-17T08:21:26.953723+0000","last_peered":"2024-09-17T08:21:26.953723+0000","last_clean":"2024-09-17T08:21:26.953723+0000","last_became_active":"2024-09-17T08:21:25.778362+0000","last_became_peered":"2024-09-17T08:21:25.778362+0000","last_unstale":"2024-09-17T08:21:26.953723+0000","last_undegraded":"2024-09-17T08:21:26.953723+0000","last_fullsized":"2024-09-17T08:21:26.953723+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:58:39.289050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.954043+0000","last_change":"2024-09-17T08:21:25.779584+0000","last_active":"2024-09-17T08:21:26.954043+0000","last_peered":"2024-09-17T08:21:26.954043+0000","last_clean":"2024-09-17T08:21:26.954043+0000","last_became_active":"2024-09-17T08:21:25.779282+0000","last_became_peered":"2024-09-17T08:21:25.779282+0000","last_unstale":"2024-09-17T08:21:26.954043+0000","last_undegraded":"2024-09-17T08:21:26.954043+0000","last_fullsized":"2024-09-17T08:21:26.954043+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:41.234351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"30'15","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.119199+0000","last_change":"2024-09-17T08:21:21.760534+0000","last_active":"2024-09-17T08:21:27.119199+0000","last_peered":"2024-09-17T08:21:27.119199+0000","last_clean":"2024-09-17T08:21:27.119199+0000","last_became_active":"2024-09-17T08:21:21.759936+0000","last_became_peered":"2024-09-17T08:21:21.759936+0000","last_unstale":"2024-09-17T08:21:27.119199+0000","last_undegraded":"2024-09-17T08:21:27.119199+0000","last_fullsized":"2024-09-17T08:21:27.119199+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:23:10.660424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.776248+0000","last_change":"2024-09-17T08:21:19.753478+0000","last_active":"2024-09-17T08:21:26.776248+0000","last_peered":"2024-09-17T08:21:26.776248+0000","last_clean":"2024-09-17T08:21:26.776248+0000","last_became_active":"2024-09-17T08:21:19.753189+0000","last_became_peered":"2024-09-17T08:21:19.753189+0000","last_unstale":"2024-09-17T08:21:26.776248+0000","last_undegraded":"2024-09-17T08:21:26.776248+0000","last_fullsized":"2024-09-17T08:21:26.776248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:04.063852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777884+0000","last_change":"2024-09-17T08:21:23.768003+0000","last_active":"2024-09-17T08:21:26.777884+0000","last_peered":"2024-09-17T08:21:26.777884+0000","last_clean":"2024-09-17T08:21:26.777884+0000","last_became_active":"2024-09-17T08:21:23.767624+0000","last_became_peered":"2024-09-17T08:21:23.767624+0000","last_unstale":"2024-09-17T08:21:26.777884+0000","last_undegraded":"2024-09-17T08:21:26.777884+0000","last_fullsized":"2024-09-17T08:21:26.777884+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:07.264004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:29.452804+0000","last_change":"2024-09-17T08:21:25.781935+0000","last_active":"2024-09-17T08:21:29.452804+0000","last_peered":"2024-09-17T08:21:29.452804+0000","last_clean":"2024-09-17T08:21:29.452804+0000","last_became_active":"2024-09-17T08:21:25.781627+0000","last_became_peered":"2024-09-17T08:21:25.781627+0000","last_unstale":"2024-09-17T08:21:29.452804+0000","last_undegraded":"2024-09-17T08:21:29.452804+0000","last_fullsized":"2024-09-17T08:21:29.452804+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:04:37.550722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.910065+0000","last_change":"2024-09-17T08:21:21.762101+0000","last_active":"2024-09-17T08:21:26.910065+0000","last_peered":"2024-09-17T08:21:26.910065+0000","last_clean":"2024-09-17T08:21:26.910065+0000","last_became_active":"2024-09-17T08:21:21.761572+0000","last_became_peered":"2024-09-17T08:21:21.761572+0000","last_unstale":"2024-09-17T08:21:26.910065+0000","last_undegraded":"2024-09-17T08:21:26.910065+0000","last_fullsized":"2024-09-17T08:21:26.910065+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:09:55.503404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.775519+0000","last_change":"2024-09-17T08:21:19.754050+0000","last_active":"2024-09-17T08:21:26.775519+0000","last_peered":"2024-09-17T08:21:26.775519+0000","last_clean":"2024-09-17T08:21:26.775519+0000","last_became_active":"2024-09-17T08:21:19.753088+0000","last_became_peered":"2024-09-17T08:21:19.753088+0000","last_unstale":"2024-09-17T08:21:26.775519+0000","last_undegraded":"2024-09-17T08:21:26.775519+0000","last_fullsized":"2024-09-17T08:21:26.775519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:10.277650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'8","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T08:21:52.900789+0000","last_change":"2024-09-17T08:21:23.767416+0000","last_active":"2024-09-17T08:21:52.900789+0000","last_peered":"2024-09-17T08:21:52.900789+0000","last_clean":"2024-09-17T08:21:52.900789+0000","last_became_active":"2024-09-17T08:21:23.767108+0000","last_became_peered":"2024-09-17T08:21:23.767108+0000","last_unstale":"2024-09-17T08:21:52.900789+0000","last_undegraded":"2024-09-17T08:21:52.900789+0000","last_fullsized":"2024-09-17T08:21:52.900789+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:39.626106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.779121+0000","last_change":"2024-09-17T08:21:19.753525+0000","last_active":"2024-09-17T08:21:26.779121+0000","last_peered":"2024-09-17T08:21:26.779121+0000","last_clean":"2024-09-17T08:21:26.779121+0000","last_became_active":"2024-09-17T08:21:19.752975+0000","last_became_peered":"2024-09-17T08:21:19.752975+0000","last_unstale":"2024-09-17T08:21:26.779121+0000","last_undegraded":"2024-09-17T08:21:26.779121+0000","last_fullsized":"2024-09-17T08:21:26.779121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:20:29.857759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"30'5","reported_seq":26,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.992232+0000","last_change":"2024-09-17T08:21:21.760487+0000","last_active":"2024-09-17T08:21:26.992232+0000","last_peered":"2024-09-17T08:21:26.992232+0000","last_clean":"2024-09-17T08:21:26.992232+0000","last_became_active":"2024-09-17T08:21:21.759984+0000","last_became_peered":"2024-09-17T08:21:21.759984+0000","last_unstale":"2024-09-17T08:21:26.992232+0000","last_undegraded":"2024-09-17T08:21:26.992232+0000","last_fullsized":"2024-09-17T08:21:26.992232+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:42:48.116537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.778667+0000","last_change":"2024-09-17T08:21:23.768298+0000","last_active":"2024-09-17T08:21:26.778667+0000","last_peered":"2024-09-17T08:21:26.778667+0000","last_clean":"2024-09-17T08:21:26.778667+0000","last_became_active":"2024-09-17T08:21:23.768009+0000","last_became_peered":"2024-09-17T08:21:23.768009+0000","last_unstale":"2024-09-17T08:21:26.778667+0000","last_undegraded":"2024-09-17T08:21:26.778667+0000","last_fullsized":"2024-09-17T08:21:26.778667+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:38.554959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:29.452756+0000","last_change":"2024-09-17T08:21:25.780375+0000","last_active":"2024-09-17T08:21:29.452756+0000","last_peered":"2024-09-17T08:21:29.452756+0000","last_clean":"2024-09-17T08:21:29.452756+0000","last_became_active":"2024-09-17T08:21:25.779669+0000","last_became_peered":"2024-09-17T08:21:25.779669+0000","last_unstale":"2024-09-17T08:21:29.452756+0000","last_undegraded":"2024-09-17T08:21:29.452756+0000","last_fullsized":"2024-09-17T08:21:29.452756+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:29.469839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.777234+0000","last_change":"2024-09-17T08:21:19.753621+0000","last_active":"2024-09-17T08:21:26.777234+0000","last_peered":"2024-09-17T08:21:26.777234+0000","last_clean":"2024-09-17T08:21:26.777234+0000","last_became_active":"2024-09-17T08:21:19.752767+0000","last_became_peered":"2024-09-17T08:21:19.752767+0000","last_unstale":"2024-09-17T08:21:26.777234+0000","last_undegraded":"2024-09-17T08:21:26.777234+0000","last_fullsized":"2024-09-17T08:21:26.777234+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:18.741686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.741686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:01.863983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:27.015134+0000","last_change":"2024-09-17T08:21:21.760582+0000","last_active":"2024-09-17T08:21:27.015134+0000","last_peered":"2024-09-17T08:21:27.015134+0000","last_clean":"2024-09-17T08:21:27.015134+0000","last_became_active":"2024-09-17T08:21:21.760072+0000","last_became_peered":"2024-09-17T08:21:21.760072+0000","last_unstale":"2024-09-17T08:21:27.015134+0000","last_undegraded":"2024-09-17T08:21:27.015134+0000","last_fullsized":"2024-09-17T08:21:27.015134+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:20.745027+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.745027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:00.017875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.778534+0000","last_change":"2024-09-17T08:21:23.764650+0000","last_active":"2024-09-17T08:21:26.778534+0000","last_peered":"2024-09-17T08:21:26.778534+0000","last_clean":"2024-09-17T08:21:26.778534+0000","last_became_active":"2024-09-17T08:21:23.764379+0000","last_became_peered":"2024-09-17T08:21:23.764379+0000","last_unstale":"2024-09-17T08:21:26.778534+0000","last_undegraded":"2024-09-17T08:21:26.778534+0000","last_fullsized":"2024-09-17T08:21:26.778534+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:22.751122+0000","last_clean_scrub_stamp":"2024-09-17T08:21:22.751122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:01.400596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T08:21:26.953958+0000","last_change":"2024-09-17T08:21:25.777731+0000","last_active":"2024-09-17T08:21:26.953958+0000","last_peered":"2024-09-17T08:21:26.953958+0000","last_clean":"2024-09-17T08:21:26.953958+0000","last_became_active":"2024-09-17T08:21:25.777427+0000","last_became_peered":"2024-09-17T08:21:25.777427+0000","last_unstale":"2024-09-17T08:21:26.953958+0000","last_undegraded":"2024-09-17T08:21:26.953958+0000","last_fullsized":"2024-09-17T08:21:26.953958+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.763085+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.763085+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:12:54.042496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":18,"seq":77309411342,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32132,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708924,"statfs":{"total":95990841344,"available":95957938176,"internally_reserved":0,"allocated":1044480,"data_stored":699977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32136,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708920,"statfs":{"total":95990841344,"available":95957934080,"internally_reserved":0,"allocated":1044480,"data_stored":699977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:21:43 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.435},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.49199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738392,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":36232,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93704824,"statfs":{"total":95990841344,"available":95953739776,"internally_reserved":0,"allocated":1044480,"data_stored":699977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:21:42 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.52000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:22:07.752 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-17T08:22:07.752 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T08:22:07.752 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-17T08:22:07.752 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-17T08:22:07.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:07 smithi193 ceph-mon[30002]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:22:07.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:07 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:07.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:07 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:07.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:07 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:07.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:07 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:07.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:07 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:07.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:07 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:07.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:07 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T08:22:07.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:07 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:07.953 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:08.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:07 smithi203 ceph-mon[30259]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:22:08.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:07 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:08.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:07 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:08.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:07 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:08.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:07 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:08.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:07 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:08.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:07 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:08.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:07 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T08:22:08.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:07 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:08.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:08 smithi203 ceph-mon[30259]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:08.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:08 smithi203 ceph-mon[30259]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-17T08:22:08.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:08 smithi203 ceph-mon[30259]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T08:22:08.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:08 smithi203 ceph-mon[30259]: Deploying daemon grafana.a on smithi193 2024-09-17T08:22:08.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:08 smithi047 ceph-mon[25866]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:08.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:08 smithi047 ceph-mon[25866]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-17T08:22:08.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:08 smithi047 ceph-mon[25866]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T08:22:08.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:08 smithi047 ceph-mon[25866]: Deploying daemon grafana.a on smithi193 2024-09-17T08:22:08.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:08 smithi193 ceph-mon[30002]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:08.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:08 smithi193 ceph-mon[30002]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-17T08:22:08.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:08 smithi193 ceph-mon[30002]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T08:22:08.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:08 smithi193 ceph-mon[30002]: Deploying daemon grafana.a on smithi193 2024-09-17T08:22:09.484 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:22:09 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-alertmanager-a[55876]: ts=2024-09-17T08:22:09.176Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000592575s 2024-09-17T08:22:09.706 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-17T08:22:09.706 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T08:22:10.295 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-17T08:22:10.295 INFO:tasks.cephadm:Setup complete, yielding 2024-09-17T08:22:10.295 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-17T08:22:10.308 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-09-17T08:22:10.308 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- bash -c 'stat -c '"'"'%u %g'"'"' /var/log/ceph | grep '"'"'167 167'"'"'' 2024-09-17T08:22:10.510 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:10.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:10 smithi193 ceph-mon[30002]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:10.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:10 smithi193 ceph-mon[30002]: from='client.? 172.21.15.47:0/2497929569' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-17T08:22:10.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:10 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:10.898 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:10 smithi047 ceph-mon[25866]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:10.898 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:10 smithi047 ceph-mon[25866]: from='client.? 172.21.15.47:0/2497929569' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-17T08:22:10.898 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:10 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:11.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:10 smithi203 ceph-mon[30259]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:11.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:10 smithi203 ceph-mon[30259]: from='client.? 172.21.15.47:0/2497929569' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-17T08:22:11.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:10 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:12.037 INFO:teuthology.orchestra.run.smithi047.stdout:167 167 2024-09-17T08:22:12.608 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-17T08:22:12.635 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:12 smithi047 ceph-mon[25866]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:12.814 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:12.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:12 smithi193 ceph-mon[30002]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:13.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:12 smithi203 ceph-mon[30259]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:14.539 INFO:teuthology.orchestra.run.smithi047.stdout:Backend: cephadm 2024-09-17T08:22:14.540 INFO:teuthology.orchestra.run.smithi047.stdout:Available: Yes 2024-09-17T08:22:14.540 INFO:teuthology.orchestra.run.smithi047.stdout:Paused: No 2024-09-17T08:22:14.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:14 smithi047 ceph-mon[25866]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:14.790 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:22:14 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: ::ffff:172.21.15.193 - - [17/Sep/2024:08:22:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:22:14.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:14 smithi193 ceph-mon[30002]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:15.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:14 smithi203 ceph-mon[30259]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:15.179 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-17T08:22:15.424 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:15.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:15 smithi193 ceph-mon[30002]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:22:15.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:15 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:22:15.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:15 smithi047 ceph-mon[25866]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:22:15.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:15 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:22:16.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:15 smithi203 ceph-mon[30259]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:22:16.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:15 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:22:16.729 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:16 smithi047 ceph-mon[25866]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:16.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:16 smithi193 ceph-mon[30002]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:17.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:16 smithi203 ceph-mon[30259]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:17.139 INFO:teuthology.orchestra.run.smithi047.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T08:22:17.139 INFO:teuthology.orchestra.run.smithi047.stdout:alertmanager.a smithi047 *:9093,9094 starting - - - - 2024-09-17T08:22:17.140 INFO:teuthology.orchestra.run.smithi047.stdout:mgr.x smithi193 *:8443,8765 running (2m) 29s ago 2m 436M - 18.2.4-855-g19e2a2be 980a0bfcd684 d54a413ea2d5 2024-09-17T08:22:17.140 INFO:teuthology.orchestra.run.smithi047.stdout:mgr.y smithi047 *:9283,8765 running (4m) 24s ago 4m 468M - 18.2.4-855-g19e2a2be 980a0bfcd684 e29578f4bd1a 2024-09-17T08:22:17.140 INFO:teuthology.orchestra.run.smithi047.stdout:mgr.z smithi203 *:8443,8765 running (2m) 30s ago 2m 436M - 18.2.4-855-g19e2a2be 980a0bfcd684 66a90b7fef04 2024-09-17T08:22:17.140 INFO:teuthology.orchestra.run.smithi047.stdout:mon.a smithi047 running (4m) 24s ago 4m 49.8M 2048M 18.2.4-855-g19e2a2be 980a0bfcd684 a0d16c90f085 2024-09-17T08:22:17.140 INFO:teuthology.orchestra.run.smithi047.stdout:mon.b smithi193 running (2m) 29s ago 2m 45.2M 2048M 18.2.4-855-g19e2a2be 980a0bfcd684 cc6a76045055 2024-09-17T08:22:17.140 INFO:teuthology.orchestra.run.smithi047.stdout:mon.c smithi203 running (2m) 30s ago 2m 44.4M 2048M 18.2.4-855-g19e2a2be 980a0bfcd684 66cd9b73a652 2024-09-17T08:22:17.140 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0 smithi047 running (2m) 24s ago 2m 58.2M 15.6G 18.2.4-855-g19e2a2be 980a0bfcd684 34bb02166ca6 2024-09-17T08:22:17.140 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1 smithi193 running (101s) 29s ago 101s 57.9M 15.6G 18.2.4-855-g19e2a2be 980a0bfcd684 d0e3c5600071 2024-09-17T08:22:17.140 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2 smithi203 running (79s) 30s ago 79s 58.4M 16.6G 18.2.4-855-g19e2a2be 980a0bfcd684 7dbf3d9cd87f 2024-09-17T08:22:17.140 INFO:teuthology.orchestra.run.smithi047.stdout:prometheus.a smithi193 *:9095 running (38s) 29s ago 38s 25.8M - 2.43.0 a07b618ecd1d 1ba33cd85096 2024-09-17T08:22:17.140 INFO:teuthology.orchestra.run.smithi047.stdout:rgw.foo.a smithi047 *:80 running (59s) 24s ago 59s 74.2M - 18.2.4-855-g19e2a2be 980a0bfcd684 f02521794540 2024-09-17T08:22:17.423 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:22:17 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-alertmanager-a[55876]: ts=2024-09-17T08:22:17.178Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003013136s 2024-09-17T08:22:17.731 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-17T08:22:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:18.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:18 smithi047 ceph-mon[25866]: from='client.14520 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:22:18.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:18 smithi047 ceph-mon[25866]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:18.815 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:18 smithi193 ceph-mon[30002]: from='client.14520 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:22:18.815 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:18 smithi193 ceph-mon[30002]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:19.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:18 smithi203 ceph-mon[30259]: from='client.14520 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:22:19.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:18 smithi203 ceph-mon[30259]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:19.144 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 podman[42638]: 2024-09-17 08:22:19.024253177 +0000 UTC m=+0.113132565 container create fe8eb604572a9f96ee8237a7444169db6f80fe347d920b61f74f0e44f0f5d9c6 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a, build-date=2022-04-29T11:48:11.055897, maintainer=Paul Cuzner , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, distribution-scope=public, architecture=x86_64, io.openshift.expose-services=, vendor=Red Hat, Inc., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., name=ubi8, version=8.5, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.tags=base rhel8, description=Ceph Grafana Container, io.buildah.version=1.32.0, vcs-type=git, com.redhat.component=ubi8-container) 2024-09-17T08:22:19.529 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 podman[42638]: 2024-09-17 08:22:19.276601315 +0000 UTC m=+0.365480714 container init fe8eb604572a9f96ee8237a7444169db6f80fe347d920b61f74f0e44f0f5d9c6 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a, build-date=2022-04-29T11:48:11.055897, release=239.1651231664, maintainer=Paul Cuzner , distribution-scope=public, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.openshift.tags=base rhel8, description=Ceph Grafana Container, io.buildah.version=1.32.0, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, architecture=x86_64, vendor=Red Hat, Inc., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, version=8.5, io.openshift.expose-services=, vcs-type=git, com.redhat.component=ubi8-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., name=ubi8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56) 2024-09-17T08:22:19.529 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 podman[42638]: 2024-09-17 08:22:19.281427966 +0000 UTC m=+0.370307355 container start fe8eb604572a9f96ee8237a7444169db6f80fe347d920b61f74f0e44f0f5d9c6 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, version=8.5, io.k8s.display-name=Red Hat Universal Base Image 8, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.component=ubi8-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., distribution-scope=public, name=ubi8, build-date=2022-04-29T11:48:11.055897, architecture=x86_64, vendor=Red Hat, Inc., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.tags=base rhel8, maintainer=Paul Cuzner , vcs-type=git, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, release=239.1651231664, description=Ceph Grafana Container, io.buildah.version=1.32.0, io.openshift.expose-services=) 2024-09-17T08:22:19.529 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 bash[42638]: fe8eb604572a9f96ee8237a7444169db6f80fe347d920b61f74f0e44f0f5d9c6 2024-09-17T08:22:19.529 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 systemd[1]: Started Ceph grafana.a for 3a66518a-74cd-11ef-bceb-c7b262605968. 2024-09-17T08:22:19.529 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=settings t=2024-09-17T08:22:19.4406546Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-17T08:22:19.529 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=settings t=2024-09-17T08:22:19.440858923Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-17T08:22:19.529 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=settings t=2024-09-17T08:22:19.440890074Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-17T08:22:19.529 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=settings t=2024-09-17T08:22:19.44090421Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-17T08:22:19.530 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=settings t=2024-09-17T08:22:19.440917546Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-17T08:22:19.530 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=settings t=2024-09-17T08:22:19.440928363Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-17T08:22:19.530 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=settings t=2024-09-17T08:22:19.440938995Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-17T08:22:19.530 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=settings t=2024-09-17T08:22:19.440949922Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-17T08:22:19.530 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=settings t=2024-09-17T08:22:19.440959741Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-17T08:22:19.530 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=settings t=2024-09-17T08:22:19.440969302Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-17T08:22:19.530 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=settings t=2024-09-17T08:22:19.440979273Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-17T08:22:19.530 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=settings t=2024-09-17T08:22:19.440989205Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-17T08:22:19.530 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=settings t=2024-09-17T08:22:19.44099963Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-17T08:22:19.530 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=settings t=2024-09-17T08:22:19.44100963Z level=info msg="App mode production" 2024-09-17T08:22:19.530 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=sqlstore t=2024-09-17T08:22:19.441072314Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-17T08:22:19.531 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=sqlstore t=2024-09-17T08:22:19.441098389Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-17T08:22:19.531 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.442249711Z level=info msg="Starting DB migrations" 2024-09-17T08:22:19.531 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.442349159Z level=info msg="Executing migration" id="create migration_log table" 2024-09-17T08:22:19.531 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.442878155Z level=info msg="Executing migration" id="create user table" 2024-09-17T08:22:19.531 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.443374046Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-17T08:22:19.531 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.444250398Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-17T08:22:19.531 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.444710765Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-17T08:22:19.531 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.447009396Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-17T08:22:19.532 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.447643777Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-17T08:22:19.532 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.448623401Z level=info msg="Executing migration" id="create user table v2" 2024-09-17T08:22:19.532 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.449154792Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-17T08:22:19.533 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.449662896Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-17T08:22:19.533 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.450086354Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-17T08:22:19.533 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.450428519Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-17T08:22:19.533 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.450758305Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-17T08:22:19.533 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.451169922Z level=info msg="Executing migration" id="Update user table charset" 2024-09-17T08:22:19.533 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.45145795Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-17T08:22:19.533 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.451831626Z level=info msg="Executing migration" id="Add missing user data" 2024-09-17T08:22:19.533 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.45218976Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-17T08:22:19.533 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.452570628Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-17T08:22:19.533 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.452937349Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-17T08:22:19.533 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.453345763Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-17T08:22:19.533 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.454438379Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-17T08:22:19.533 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.454887002Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-17T08:22:19.534 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.455342689Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-17T08:22:19.534 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.455800345Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-17T08:22:19.534 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.458430413Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-17T08:22:19.534 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.461203229Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-17T08:22:19.534 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.463615918Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-17T08:22:19.534 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.466429223Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-17T08:22:19.534 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.468792029Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-17T08:22:19.534 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.471445994Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-17T08:22:19.534 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.471943525Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-17T08:22:19.534 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.474935874Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-17T08:22:19.534 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.477442237Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-17T08:22:19.534 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.480040144Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-17T08:22:19.534 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.480432541Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-17T08:22:19.535 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.48301827Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-17T08:22:19.535 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.483432891Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-17T08:22:19.535 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.485607557Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-17T08:22:19.535 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.486060815Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-17T08:22:19.535 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.488264845Z level=info msg="Executing migration" id="create star table" 2024-09-17T08:22:19.535 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.489060633Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-17T08:22:19.535 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.489790408Z level=info msg="Executing migration" id="create org table v1" 2024-09-17T08:22:19.535 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.490561166Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-17T08:22:19.535 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.49099513Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-17T08:22:19.535 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.491796638Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-17T08:22:19.536 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.492621086Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-17T08:22:19.536 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.493072611Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-17T08:22:19.536 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.49348734Z level=info msg="Executing migration" id="Update org table charset" 2024-09-17T08:22:19.536 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.494234452Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-17T08:22:19.536 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.494575441Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-17T08:22:19.536 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.495109137Z level=info msg="Executing migration" id="create dashboard table" 2024-09-17T08:22:19.536 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.495761083Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-17T08:22:19.537 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.496190457Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-17T08:22:19.537 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.496976563Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-17T08:22:19.537 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.49787128Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-17T08:22:19.537 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.498790104Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-17T08:22:19.537 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.499423139Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-17T08:22:19.537 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.500305365Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-17T08:22:19.537 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.501134085Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-17T08:22:19.537 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.502002346Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-17T08:22:19.537 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.502767884Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-17T08:22:19.537 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.503673214Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-17T08:22:19.537 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.504329204Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-17T08:22:19.537 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.505146327Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-17T08:22:19.537 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.505941725Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-17T08:22:19.538 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.506755875Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-17T08:22:19.538 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.507596282Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-17T08:22:19.538 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.508575243Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-17T08:22:19.538 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.509317109Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-17T08:22:19.538 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.510108047Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-17T08:22:19.538 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.510969452Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-17T08:22:19.538 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.511956292Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-17T08:22:19.538 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.512657334Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-17T08:22:19.538 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.513406029Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-17T08:22:19.538 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.513953172Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-17T08:22:19.538 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.514728119Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-17T08:22:19.538 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.517624467Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-17T08:22:19.538 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.518067696Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-17T08:22:19.538 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.521051Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-17T08:22:19.539 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.52375297Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-17T08:22:19.539 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.526567145Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-17T08:22:19.539 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.529169881Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-17T08:22:19.755 INFO:teuthology.orchestra.run.smithi047.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-17T08:22:19.755 INFO:teuthology.orchestra.run.smithi047.stdout:alertmanager ?:9093,9094 0/1 - 57s smithi047=a;count:1 2024-09-17T08:22:19.755 INFO:teuthology.orchestra.run.smithi047.stdout:grafana ?:3000 0/1 - 54s smithi193=a;count:1 2024-09-17T08:22:19.755 INFO:teuthology.orchestra.run.smithi047.stdout:mgr 3/3 33s ago 2m smithi047=y;smithi193=x;smithi203=z;count:3 2024-09-17T08:22:19.755 INFO:teuthology.orchestra.run.smithi047.stdout:mon 3/3 33s ago 2m smithi047:172.21.15.47=a;smithi193:172.21.15.193=b;smithi203:172.21.15.203=c;count:3 2024-09-17T08:22:19.755 INFO:teuthology.orchestra.run.smithi047.stdout:node-exporter ?:9100 0/3 - 59s smithi047=a;smithi193=b;smithi203=c;count:3 2024-09-17T08:22:19.755 INFO:teuthology.orchestra.run.smithi047.stdout:osd 3 33s ago - 2024-09-17T08:22:19.756 INFO:teuthology.orchestra.run.smithi047.stdout:prometheus ?:9095 1/1 32s ago 62s smithi193=a;count:1 2024-09-17T08:22:19.756 INFO:teuthology.orchestra.run.smithi047.stdout:rgw.foo.a ?:80 1/1 27s ago 61s smithi047=foo.a;count:1 2024-09-17T08:22:19.781 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.531872583Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-17T08:22:19.781 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.534459384Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-17T08:22:19.781 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.537169122Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-17T08:22:19.781 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.539960788Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-17T08:22:19.781 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.542591178Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-17T08:22:19.781 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.545171031Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-17T08:22:19.781 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.54742291Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-17T08:22:19.781 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.550076431Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-17T08:22:19.782 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.552841545Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-17T08:22:19.782 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.555648095Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-17T08:22:19.782 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.558010748Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-17T08:22:19.782 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.560702539Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-17T08:22:19.782 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.563628635Z level=info msg="Executing migration" id="create data_source table" 2024-09-17T08:22:19.782 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.566098345Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-17T08:22:19.782 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.566936514Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-17T08:22:19.782 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.567732756Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-17T08:22:19.782 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.568512457Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-17T08:22:19.782 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.569300674Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-17T08:22:19.782 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.570398157Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-17T08:22:19.782 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.571214268Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-17T08:22:19.782 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.57198512Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-17T08:22:19.783 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.572988086Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-17T08:22:19.783 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.573759066Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-17T08:22:19.783 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.574529792Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-17T08:22:19.783 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.57539115Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-17T08:22:19.783 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.576357549Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-17T08:22:19.783 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.577068223Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-17T08:22:19.783 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.577978073Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-17T08:22:19.783 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.57878438Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-17T08:22:19.783 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.57974327Z level=info msg="Executing migration" id="Add uid column" 2024-09-17T08:22:19.783 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.580596264Z level=info msg="Executing migration" id="Update uid value" 2024-09-17T08:22:19.783 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.581324816Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-17T08:22:19.784 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.58219043Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-17T08:22:19.784 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.583040177Z level=info msg="Executing migration" id="create api_key table" 2024-09-17T08:22:19.784 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.583896673Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-17T08:22:19.784 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.58485822Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-17T08:22:19.784 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.585638721Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-17T08:22:19.784 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.586602655Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-17T08:22:19.785 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.587323875Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-17T08:22:19.785 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.588290291Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-17T08:22:19.785 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.589013317Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-17T08:22:19.785 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.59007397Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-17T08:22:19.786 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.591077304Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-17T08:22:19.786 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.591818052Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-17T08:22:19.786 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.592833785Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-17T08:22:19.786 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.593386593Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-17T08:22:19.786 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.594257568Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-17T08:22:19.786 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.594810987Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-17T08:22:19.786 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.595149908Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-17T08:22:19.786 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.598111799Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-17T08:22:19.786 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.598804501Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-17T08:22:19.786 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.601664166Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-17T08:22:19.786 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.604449411Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-17T08:22:19.786 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.607331338Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-17T08:22:19.786 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.607757515Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-17T08:22:19.786 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.610546824Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-17T08:22:19.787 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.613338062Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-17T08:22:19.787 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.616021247Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-17T08:22:19.787 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.616490136Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-17T08:22:19.787 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.619268749Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-17T08:22:19.787 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.621966079Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-17T08:22:19.787 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.625014376Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-17T08:22:19.787 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.625743082Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-17T08:22:19.787 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.628572616Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-17T08:22:19.787 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.631474273Z level=info msg="Executing migration" id="create quota table v1" 2024-09-17T08:22:19.787 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.634176597Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-17T08:22:19.787 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.634680147Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-17T08:22:19.787 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.637164983Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-17T08:22:19.787 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.637606021Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-17T08:22:19.788 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.640044476Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-17T08:22:19.788 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.642580715Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-17T08:22:19.788 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.645301059Z level=info msg="Executing migration" id="create session table" 2024-09-17T08:22:19.788 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.647753497Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-17T08:22:19.788 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.650567851Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-17T08:22:19.788 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.653333325Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-17T08:22:19.788 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.653799548Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-17T08:22:19.788 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.656570695Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-17T08:22:19.788 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.659085666Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-17T08:22:19.788 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.661916552Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-17T08:22:19.788 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.662391359Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-17T08:22:19.788 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.66511613Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-17T08:22:19.788 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.665573174Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-17T08:22:19.789 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.667866536Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-17T08:22:19.789 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.66853814Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-17T08:22:19.789 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.670374638Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-17T08:22:19.789 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.671035377Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-17T08:22:19.789 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.671656965Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-17T08:22:19.789 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.672445806Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-17T08:22:19.789 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.672919923Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-17T08:22:19.789 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.673393993Z level=info msg="Executing migration" id="create alert table v1" 2024-09-17T08:22:19.789 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.674191232Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-17T08:22:19.789 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.674670131Z level=info msg="Executing migration" id="add index alert state" 2024-09-17T08:22:19.789 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.675644775Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-17T08:22:19.789 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.676252259Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-17T08:22:19.789 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.677097416Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-17T08:22:19.789 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.677535839Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-17T08:22:19.790 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.67835418Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-17T08:22:19.790 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.67982425Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-17T08:22:19.790 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.680237051Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-17T08:22:19.790 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.681106749Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-17T08:22:19.790 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.681469707Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-17T08:22:19.790 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.682401344Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-17T08:22:19.790 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.682973011Z level=info msg="Executing migration" id="Add column is_default" 2024-09-17T08:22:19.790 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.683636341Z level=info msg="Executing migration" id="Add column frequency" 2024-09-17T08:22:19.790 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.684493754Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-17T08:22:19.790 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.685184688Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-17T08:22:19.790 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.686056508Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-17T08:22:19.790 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.687021707Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-17T08:22:19.790 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.687736455Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-17T08:22:19.790 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.688087708Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-17T08:22:19.791 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.689104451Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-17T08:22:19.791 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.689791723Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-17T08:22:19.791 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.690629046Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-17T08:22:19.791 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.691098022Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-17T08:22:19.791 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.692091438Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-17T08:22:19.791 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.69292034Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-17T08:22:19.791 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.693732664Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-17T08:22:19.791 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.694085879Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-17T08:22:19.791 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.694975056Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-17T08:22:19.791 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.697772196Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-17T08:22:19.791 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.700605076Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-17T08:22:19.791 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.700978151Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-17T08:22:19.791 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.703826322Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-17T08:22:19.791 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.706575176Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-17T08:22:19.792 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.709271919Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-17T08:22:19.792 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.709833642Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-17T08:22:19.792 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.712733191Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-17T08:22:19.792 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.715442502Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-17T08:22:19.792 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.718140263Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-17T08:22:19.792 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.718636969Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-17T08:22:19.792 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.719456696Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-17T08:22:19.792 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.720233296Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-17T08:22:19.792 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.721009245Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-17T08:22:19.792 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.721455325Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-17T08:22:19.792 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.72223155Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-17T08:22:19.792 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.723016283Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-17T08:22:19.792 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.724043175Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-17T08:22:19.792 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.724577494Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-17T08:22:19.793 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.72744129Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-17T08:22:19.793 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.730015636Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-17T08:22:19.793 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.732743786Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-17T08:22:19.793 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.733130379Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-17T08:22:19.793 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.735886373Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-17T08:22:19.793 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.738642199Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-17T08:22:19.793 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.741104463Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-17T08:22:19.793 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.741837485Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-17T08:22:19.793 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.744637718Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-17T08:22:19.793 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.74731894Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-17T08:22:19.793 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.749752928Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-17T08:22:19.793 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.750534958Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-17T08:22:19.793 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.753258453Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-17T08:22:19.793 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.7561639Z level=info msg="Executing migration" id="Move region to single row" 2024-09-17T08:22:19.794 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.758965286Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-17T08:22:19.794 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.759533761Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-17T08:22:19.794 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.762306402Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-17T08:22:19.794 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.764712515Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-17T08:22:19.794 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.767467314Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-17T08:22:19.794 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.768162815Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-17T08:22:19.794 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.770739332Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-17T08:22:19.794 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.773311996Z level=info msg="Executing migration" id="create test_data table" 2024-09-17T08:22:19.794 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.773745623Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-17T08:22:19.794 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.776018932Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-17T08:22:19.794 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.778558155Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-17T08:22:19.794 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.781029201Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-17T08:22:19.794 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.783538326Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-17T08:22:19.794 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.783911363Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-17T08:22:20.031 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.786624555Z level=info msg="Executing migration" id="create team table" 2024-09-17T08:22:20.031 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.789266987Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-17T08:22:20.032 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.791872493Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-17T08:22:20.032 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.792393702Z level=info msg="Executing migration" id="create team member table" 2024-09-17T08:22:20.032 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.79497426Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-17T08:22:20.032 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.797671166Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-17T08:22:20.032 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.800268042Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-17T08:22:20.032 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.800864698Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-17T08:22:20.032 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.801650083Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-17T08:22:20.032 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.802421178Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-17T08:22:20.032 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.803228446Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-17T08:22:20.032 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.803695808Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-17T08:22:20.032 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.804410607Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-17T08:22:20.032 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.805133114Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-17T08:22:20.033 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.805603109Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-17T08:22:20.033 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.80608467Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-17T08:22:20.033 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.806755698Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-17T08:22:20.033 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.807225849Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-17T08:22:20.033 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.80765873Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-17T08:22:20.033 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.808092295Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-17T08:22:20.033 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.808421656Z level=info msg="Executing migration" id="create tag table" 2024-09-17T08:22:20.033 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.808820896Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-17T08:22:20.033 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.809276067Z level=info msg="Executing migration" id="create login attempt table" 2024-09-17T08:22:20.033 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.809678797Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-17T08:22:20.033 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.810149677Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-17T08:22:20.033 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.810602214Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-17T08:22:20.033 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.812489168Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-17T08:22:20.034 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.812928811Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-17T08:22:20.034 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.813398526Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-17T08:22:20.034 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.813708974Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-17T08:22:20.034 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.814125497Z level=info msg="Executing migration" id="create user auth table" 2024-09-17T08:22:20.034 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.81455126Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-17T08:22:20.034 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.81504764Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-17T08:22:20.034 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.815362807Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-17T08:22:20.034 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.816178363Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-17T08:22:20.035 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.816956332Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-17T08:22:20.035 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.81774775Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-17T08:22:20.035 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.818542822Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-17T08:22:20.036 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.819026156Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-17T08:22:20.037 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.819808717Z level=info msg="Executing migration" id="create server_lock table" 2024-09-17T08:22:20.037 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.820281871Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-17T08:22:20.037 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.820769867Z level=info msg="Executing migration" id="create user auth token table" 2024-09-17T08:22:20.037 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.821280311Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-17T08:22:20.037 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.821762816Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-17T08:22:20.037 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.822235477Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-17T08:22:20.037 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.822714079Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-17T08:22:20.037 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.823554989Z level=info msg="Executing migration" id="create cache_data table" 2024-09-17T08:22:20.037 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.824011292Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-17T08:22:20.037 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.824480278Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-17T08:22:20.037 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.824960258Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-17T08:22:20.037 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.825619982Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-17T08:22:20.038 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.825919085Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-17T08:22:20.038 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.826257605Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-17T08:22:20.038 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.826731805Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-17T08:22:20.038 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.827535721Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-17T08:22:20.038 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.828154772Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-17T08:22:20.038 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.828476798Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-17T08:22:20.038 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.828910616Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-17T08:22:20.038 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.829366959Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-17T08:22:20.038 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.829789142Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-17T08:22:20.038 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.830241192Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-17T08:22:20.038 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.831072464Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-17T08:22:20.038 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.831852071Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-17T08:22:20.038 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.832806138Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-17T08:22:20.039 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.833451327Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-17T08:22:20.039 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.834303999Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-17T08:22:20.039 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.834980393Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-17T08:22:20.039 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.835411483Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-17T08:22:20.039 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.836084149Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-17T08:22:20.039 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.836923155Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-17T08:22:20.039 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.837761959Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-17T08:22:20.039 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.838496348Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-17T08:22:20.039 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.839321145Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-17T08:22:20.039 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.840112982Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-17T08:22:20.039 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.841017773Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-17T08:22:20.040 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.844507272Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-17T08:22:20.040 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.847876384Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-17T08:22:20.040 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.848660978Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-17T08:22:20.040 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.849449845Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-17T08:22:20.040 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.850280159Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-17T08:22:20.040 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.852404706Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-17T08:22:20.040 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.855276065Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-17T08:22:20.040 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.85769255Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-17T08:22:20.040 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.860620001Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-17T08:22:20.040 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.863283132Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-17T08:22:20.040 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.865969631Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-17T08:22:20.040 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.868610572Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-17T08:22:20.040 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.871409919Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-17T08:22:20.040 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.874090001Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-17T08:22:20.041 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.876629969Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-17T08:22:20.041 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.879313115Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-17T08:22:20.041 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.882115632Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-17T08:22:20.041 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.884775231Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-17T08:22:20.041 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.887335784Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-17T08:22:20.041 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.890180419Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-17T08:22:20.041 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.892955704Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-17T08:22:20.041 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.893418412Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-17T08:22:20.041 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.896259045Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-17T08:22:20.041 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.899133137Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-17T08:22:20.041 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.90170268Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-17T08:22:20.041 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.904553546Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-17T08:22:20.041 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.907290818Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-17T08:22:20.042 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.910186515Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-17T08:22:20.042 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.912778987Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-17T08:22:20.042 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.915682313Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-17T08:22:20.042 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.918527922Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-17T08:22:20.042 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.921279996Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-17T08:22:20.042 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.924125766Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-17T08:22:20.042 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.926999236Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-17T08:22:20.042 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.92969963Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-17T08:22:20.042 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.932627412Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-17T08:22:20.042 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.935292275Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-17T08:22:20.042 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.937985974Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-17T08:22:20.042 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.940830167Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-17T08:22:20.042 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.943498584Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-17T08:22:20.042 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.946056229Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-17T08:22:20.043 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.948652263Z level=info msg="Executing migration" id="create alert_image table" 2024-09-17T08:22:20.043 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.951417929Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-17T08:22:20.043 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.953751888Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-17T08:22:20.043 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.956435785Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-17T08:22:20.043 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.95905008Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-17T08:22:20.043 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.961793302Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-17T08:22:20.043 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.961887338Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-17T08:22:20.043 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.964631565Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-17T08:22:20.043 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.96731874Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-17T08:22:20.043 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.969961128Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-17T08:22:20.043 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.970087565Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-17T08:22:20.043 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.970206201Z level=info msg="alerts found to migrate" alerts=0 2024-09-17T08:22:20.043 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.972765922Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-17T08:22:20.044 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.975391379Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-17T08:22:20.044 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.978249959Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-17T08:22:20.044 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.978964032Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-17T08:22:20.044 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.979900872Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-17T08:22:20.044 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.980614837Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-17T08:22:20.044 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.981359499Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-17T08:22:20.044 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.981766258Z level=info msg="Executing migration" id="create data_keys table" 2024-09-17T08:22:20.044 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.982252181Z level=info msg="Executing migration" id="create secrets table" 2024-09-17T08:22:20.044 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.983069305Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-17T08:22:20.044 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.987306718Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-17T08:22:20.044 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.988228271Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-17T08:22:20.044 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.989006603Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-17T08:22:20.044 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.993204612Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-17T08:22:20.044 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.997453388Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-17T08:22:20.045 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.998285441Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-17T08:22:20.045 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:19 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.999104007Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-17T08:22:20.045 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:19.999972405Z level=info msg="Executing migration" id="create permission table" 2024-09-17T08:22:20.045 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.000813675Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-17T08:22:20.045 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.001623433Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-17T08:22:20.045 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.002497771Z level=info msg="Executing migration" id="create role table" 2024-09-17T08:22:20.045 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.00352419Z level=info msg="Executing migration" id="add column display_name" 2024-09-17T08:22:20.045 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.004490276Z level=info msg="Executing migration" id="add column group_name" 2024-09-17T08:22:20.045 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.005456566Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-17T08:22:20.045 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.006333474Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-17T08:22:20.045 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.00718906Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-17T08:22:20.046 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.008044738Z level=info msg="Executing migration" id="create team role table" 2024-09-17T08:22:20.046 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.008916778Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-17T08:22:20.046 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.009855168Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-17T08:22:20.046 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.010577407Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-17T08:22:20.046 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.011595577Z level=info msg="Executing migration" id="create user role table" 2024-09-17T08:22:20.046 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.012336109Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-17T08:22:20.046 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.013184747Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-17T08:22:20.046 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.014087131Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-17T08:22:20.046 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.01507698Z level=info msg="Executing migration" id="create builtin role table" 2024-09-17T08:22:20.046 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.015792206Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-17T08:22:20.046 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.016780614Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-17T08:22:20.046 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.017528954Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-17T08:22:20.046 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.020575435Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-17T08:22:20.046 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.023246566Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-17T08:22:20.047 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.025928994Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-17T08:22:20.047 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.028575243Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-17T08:22:20.047 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.031426422Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-17T08:22:20.047 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.034232731Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-17T08:22:20.394 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.036977053Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-17T08:22:20.394 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.039757978Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-17T08:22:20.394 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.042538258Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-17T08:22:20.394 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.045071326Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-17T08:22:20.394 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.047819803Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-17T08:22:20.395 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.050627514Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-17T08:22:20.395 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.053449267Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-17T08:22:20.395 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.056543877Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-17T08:22:20.395 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.059167861Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-17T08:22:20.395 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.06200114Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-17T08:22:20.395 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.064700222Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-17T08:22:20.395 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.067636118Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-17T08:22:20.395 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.070306949Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-17T08:22:20.395 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.073102273Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-17T08:22:20.395 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.075715531Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-17T08:22:20.395 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.078335532Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-17T08:22:20.395 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.081044111Z level=info msg="Executing migration" id="add correlation config column" 2024-09-17T08:22:20.395 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.083702913Z level=info msg="Executing migration" id="create entity_events table" 2024-09-17T08:22:20.395 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.08635064Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-17T08:22:20.396 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.089185304Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-17T08:22:20.396 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.089348939Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-17T08:22:20.396 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.091724241Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-17T08:22:20.396 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.091975659Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-17T08:22:20.396 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.094405266Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-17T08:22:20.396 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.096955205Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-17T08:22:20.396 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.099604974Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-17T08:22:20.396 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.102376944Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-17T08:22:20.396 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.104957414Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-17T08:22:20.396 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.107737707Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-17T08:22:20.396 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.110356007Z level=info msg="Executing migration" id="Drop public config table" 2024-09-17T08:22:20.396 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.113049338Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-17T08:22:20.396 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.113868946Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-17T08:22:20.396 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.114584373Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-17T08:22:20.397 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.115394403Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-17T08:22:20.397 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.116236106Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-17T08:22:20.397 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.1192115Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-17T08:22:20.397 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.120264862Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-17T08:22:20.397 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.121314612Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-17T08:22:20.397 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.122217002Z level=info msg="Executing migration" id="add share column" 2024-09-17T08:22:20.398 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.123322641Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-17T08:22:20.398 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.124141031Z level=info msg="Executing migration" id="create file table" 2024-09-17T08:22:20.398 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.124993666Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-17T08:22:20.398 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.125818525Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-17T08:22:20.398 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.126694948Z level=info msg="Executing migration" id="create file_meta table" 2024-09-17T08:22:20.398 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.12768034Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-17T08:22:20.398 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.128541195Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-17T08:22:20.398 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.129277507Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-17T08:22:20.399 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.130072614Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-17T08:22:20.399 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.130930734Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-17T08:22:20.399 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.131816694Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-17T08:22:20.399 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.132867992Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-17T08:22:20.399 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.133716489Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-17T08:22:20.399 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.134673067Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-17T08:22:20.399 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.135416137Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-17T08:22:20.399 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.136222625Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-17T08:22:20.399 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.136654159Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-17T08:22:20.399 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.137729573Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-17T08:22:20.399 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.138797434Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-17T08:22:20.399 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.141213987Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-17T08:22:20.399 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.150707396Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-17T08:22:20.400 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.153297539Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-17T08:22:20.400 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.155862811Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-17T08:22:20.400 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.15930272Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-17T08:22:20.400 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.16186851Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-17T08:22:20.400 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.164630689Z level=info msg="Executing migration" id="create folder table" 2024-09-17T08:22:20.400 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.16689466Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-17T08:22:20.400 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.169334285Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-17T08:22:20.400 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.172059196Z level=info msg="Executing migration" id="Update folder title length" 2024-09-17T08:22:20.400 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.172644243Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-17T08:22:20.400 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=migrator t=2024-09-17T08:22:20.17359211Z level=info msg="migrations completed" performed=485 skipped=0 duration=731.274151ms 2024-09-17T08:22:20.400 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=sqlstore t=2024-09-17T08:22:20.174161015Z level=info msg="Created default organization" 2024-09-17T08:22:20.400 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=plugin.loader t=2024-09-17T08:22:20.230169104Z level=info msg="Plugin registered" pluginID=input 2024-09-17T08:22:20.400 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=plugin.loader t=2024-09-17T08:22:20.256488897Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-17T08:22:20.400 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=plugin.loader t=2024-09-17T08:22:20.256506995Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-17T08:22:20.401 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=secrets t=2024-09-17T08:22:20.256549725Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-17T08:22:20.401 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=query_data t=2024-09-17T08:22:20.258300106Z level=info msg="Query Service initialization" 2024-09-17T08:22:20.401 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=live.push_http t=2024-09-17T08:22:20.260592154Z level=info msg="Live Push Gateway initialization" 2024-09-17T08:22:20.420 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:20 smithi047 systemd[1]: Starting Ceph node-exporter.a for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:22:20.442 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-17T08:22:20.666 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:20.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:20 smithi203 ceph-mon[30259]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:20.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:20 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:20.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:20 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:20.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:20 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:20.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:20 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:20.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:20 smithi203 ceph-mon[30259]: Deploying daemon node-exporter.a on smithi047 2024-09-17T08:22:20.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:20 smithi203 ceph-mon[30259]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:22:20.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:20 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:20.788 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:20 smithi047 bash[57204]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-17T08:22:20.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:20 smithi047 ceph-mon[25866]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:20.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:20 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:20.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:20 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:20.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:20 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:20.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:20 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:20.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:20 smithi047 ceph-mon[25866]: Deploying daemon node-exporter.a on smithi047 2024-09-17T08:22:20.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:20 smithi047 ceph-mon[25866]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:22:20.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:20 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:20.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-mon[30002]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:20.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:20.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:20.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:20.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:20.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-mon[30002]: Deploying daemon node-exporter.a on smithi047 2024-09-17T08:22:20.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-mon[30002]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:22:20.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:21.039 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:20 smithi047 bash[57204]: Getting image source signatures 2024-09-17T08:22:21.040 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:20 smithi047 bash[57204]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-17T08:22:21.040 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:20 smithi047 bash[57204]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-17T08:22:21.040 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:20 smithi047 bash[57204]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-17T08:22:21.394 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:20 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=infra.usagestats.collector t=2024-09-17T08:22:20.971252556Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-17T08:22:21.394 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:21 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=provisioning.datasources t=2024-09-17T08:22:21.027849149Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-17T08:22:21.394 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:21 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=provisioning.datasources t=2024-09-17T08:22:21.059413418Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-17T08:22:21.394 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:21 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=provisioning.alerting t=2024-09-17T08:22:21.080865278Z level=info msg="starting to provision alerting" 2024-09-17T08:22:21.395 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:21 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=provisioning.alerting t=2024-09-17T08:22:21.080910702Z level=info msg="finished to provision alerting" 2024-09-17T08:22:21.395 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:21 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=grafanaStorageLogger t=2024-09-17T08:22:21.081020551Z level=info msg="storage starting" 2024-09-17T08:22:21.395 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:21 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=ngalert.state.manager t=2024-09-17T08:22:21.081095257Z level=info msg="Warming state cache for startup" 2024-09-17T08:22:21.395 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:21 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=http.server t=2024-09-17T08:22:21.084368033Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-17T08:22:21.395 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:21 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=ngalert.state.manager t=2024-09-17T08:22:21.091399021Z level=info msg="State cache has been initialized" states=0 duration=10.301856ms 2024-09-17T08:22:21.395 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:21 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=ticker t=2024-09-17T08:22:21.091467866Z level=info msg=starting first_tick=2024-09-17T08:22:30Z 2024-09-17T08:22:21.396 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:21 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=ngalert.multiorg.alertmanager t=2024-09-17T08:22:21.091464899Z level=info msg="starting MultiOrg Alertmanager" 2024-09-17T08:22:21.396 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:21 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=sqlstore.transactions t=2024-09-17T08:22:21.091532527Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-17T08:22:21.396 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:21 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=sqlstore.transactions t=2024-09-17T08:22:21.136114483Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-17T08:22:21.396 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:21 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=sqlstore.transactions t=2024-09-17T08:22:21.146655841Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-17T08:22:21.396 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:22:21 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a[42653]: logger=sqlstore.transactions t=2024-09-17T08:22:21.182257852Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-17T08:22:22.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:22 smithi047 ceph-mon[25866]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:22.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:22 smithi203 ceph-mon[30259]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:22.894 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:22 smithi193 ceph-mon[30002]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:23.039 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:22 smithi047 bash[57204]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-17T08:22:23.131 INFO:teuthology.orchestra.run.smithi047.stdout:HOST ADDR LABELS STATUS 2024-09-17T08:22:23.131 INFO:teuthology.orchestra.run.smithi047.stdout:smithi047 172.21.15.47 2024-09-17T08:22:23.131 INFO:teuthology.orchestra.run.smithi047.stdout:smithi193 172.21.15.193 2024-09-17T08:22:23.131 INFO:teuthology.orchestra.run.smithi047.stdout:smithi203 172.21.15.203 2024-09-17T08:22:23.131 INFO:teuthology.orchestra.run.smithi047.stdout:3 hosts in cluster 2024-09-17T08:22:23.539 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:23 smithi047 bash[57204]: Writing manifest to image destination 2024-09-17T08:22:23.850 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:23 smithi047 podman[57204]: 2024-09-17 08:22:23.515979212 +0000 UTC m=+2.981028916 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-17T08:22:23.850 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:23 smithi047 podman[57204]: 2024-09-17 08:22:23.630161525 +0000 UTC m=+3.095211227 container create a7d7148fb0b5bb585900ca783c70061e264e98ce4b205b3a6745b1b0eb9c5aba (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T08:22:24.119 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-17T08:22:24.386 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 podman[57204]: 2024-09-17 08:22:24.276863559 +0000 UTC m=+3.741913270 container init a7d7148fb0b5bb585900ca783c70061e264e98ce4b205b3a6745b1b0eb9c5aba (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T08:22:24.386 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 podman[57204]: 2024-09-17 08:22:24.284104564 +0000 UTC m=+3.749154273 container start a7d7148fb0b5bb585900ca783c70061e264e98ce4b205b3a6745b1b0eb9c5aba (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T08:22:24.386 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.288Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-17T08:22:24.386 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.288Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-17T08:22:24.386 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.289Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-17T08:22:24.386 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.289Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-17T08:22:24.386 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.289Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-17T08:22:24.386 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.289Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-17T08:22:24.386 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.289Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-17T08:22:24.387 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=arp 2024-09-17T08:22:24.387 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-17T08:22:24.387 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-17T08:22:24.387 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-17T08:22:24.387 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-17T08:22:24.387 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-17T08:22:24.387 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-17T08:22:24.387 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-17T08:22:24.388 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-17T08:22:24.388 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=edac 2024-09-17T08:22:24.388 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-17T08:22:24.388 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-17T08:22:24.388 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-17T08:22:24.388 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-17T08:22:24.388 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-17T08:22:24.388 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-17T08:22:24.388 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-17T08:22:24.388 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-17T08:22:24.388 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-17T08:22:24.388 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-17T08:22:24.388 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-17T08:22:24.388 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-17T08:22:24.389 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-17T08:22:24.389 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-17T08:22:24.389 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-17T08:22:24.389 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-17T08:22:24.389 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=os 2024-09-17T08:22:24.389 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-17T08:22:24.389 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-17T08:22:24.389 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-17T08:22:24.389 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-17T08:22:24.389 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-17T08:22:24.389 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-17T08:22:24.389 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-17T08:22:24.389 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=stat 2024-09-17T08:22:24.389 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-17T08:22:24.390 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-17T08:22:24.390 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-17T08:22:24.390 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=time 2024-09-17T08:22:24.390 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-17T08:22:24.390 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=uname 2024-09-17T08:22:24.390 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-17T08:22:24.390 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-17T08:22:24.390 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-17T08:22:24.390 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-17T08:22:24.390 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a[57467]: ts=2024-09-17T08:22:24.290Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-17T08:22:24.390 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 bash[57204]: a7d7148fb0b5bb585900ca783c70061e264e98ce4b205b3a6745b1b0eb9c5aba 2024-09-17T08:22:24.446 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:24.675 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-mon[25866]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:22:24.676 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-mon[25866]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:24.676 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:22:24 smithi047 systemd[1]: Started Ceph node-exporter.a for 3a66518a-74cd-11ef-bceb-c7b262605968. 2024-09-17T08:22:24.676 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:22:24 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: ::ffff:172.21.15.193 - - [17/Sep/2024:08:22:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:22:24.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:24 smithi203 ceph-mon[30259]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:22:24.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:24 smithi203 ceph-mon[30259]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:24.816 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:24 smithi193 ceph-mon[30002]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:22:24.816 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:24 smithi193 ceph-mon[30002]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:25.455 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:25 smithi193 bash[42918]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-17T08:22:25.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:25 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:25.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:25 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:25.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:25 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:25.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:25 smithi047 ceph-mon[25866]: Deploying daemon node-exporter.b on smithi193 2024-09-17T08:22:25.730 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:25 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:25.731 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:25 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:25.731 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:25 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:25.731 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:25 smithi193 ceph-mon[30002]: Deploying daemon node-exporter.b on smithi193 2024-09-17T08:22:25.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:25 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:25.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:25 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:25.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:25 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:25.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:25 smithi203 ceph-mon[30259]: Deploying daemon node-exporter.b on smithi193 2024-09-17T08:22:26.093 INFO:teuthology.orchestra.run.smithi047.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-17T08:22:26.093 INFO:teuthology.orchestra.run.smithi047.stdout:smithi047 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130022480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-09-17T08:22:26.093 INFO:teuthology.orchestra.run.smithi047.stdout:smithi047 /dev/nvme1n1 ssd Linux_053f7dfeea999baddc61 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-17T08:22:26.093 INFO:teuthology.orchestra.run.smithi047.stdout:smithi047 /dev/nvme2n1 ssd Linux_a2d34d393cc9b2254f8e 89.4G Yes 2m ago 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi047.stdout:smithi047 /dev/nvme3n1 ssd Linux_072b873147b212ae6904 89.4G Yes 2m ago 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi047.stdout:smithi047 /dev/nvme4n1 ssd Linux_6103c9bfef571206e4d1 89.4G Yes 2m ago 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi047.stdout:smithi193 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500HB400BGN 372G No 103s ago Has a FileSystem, LVM detected 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi047.stdout:smithi193 /dev/nvme1n1 ssd Linux_ce79d4faf185e7f09943 89.4G No 103s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi047.stdout:smithi193 /dev/nvme2n1 ssd Linux_71bb9e14d4d0843f10b1 89.4G Yes 103s ago 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi047.stdout:smithi193 /dev/nvme3n1 ssd Linux_8c2bd542fdcb63d32d20 89.4G Yes 103s ago 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi047.stdout:smithi193 /dev/nvme4n1 ssd Linux_75491bf8ac535f5fd462 89.4G Yes 103s ago 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi047.stdout:smithi203 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GK400BGN 372G No 81s ago Has a FileSystem, LVM detected 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi047.stdout:smithi203 /dev/nvme1n1 ssd Linux_265e35466071d7b72f61 89.4G No 81s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi047.stdout:smithi203 /dev/nvme2n1 ssd Linux_406c44bc223c3a803c86 89.4G Yes 81s ago 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi047.stdout:smithi203 /dev/nvme3n1 ssd Linux_038c12331e5bd189c77d 89.4G Yes 81s ago 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi047.stdout:smithi203 /dev/nvme4n1 ssd Linux_3dc0b248e4c662526a12 89.4G Yes 81s ago 2024-09-17T08:22:26.143 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:25 smithi193 bash[42918]: Getting image source signatures 2024-09-17T08:22:26.143 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:25 smithi193 bash[42918]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-17T08:22:26.144 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:25 smithi193 bash[42918]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-17T08:22:26.144 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:25 smithi193 bash[42918]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-17T08:22:26.652 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls --format yaml' 2024-09-17T08:22:26.675 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:26 smithi047 ceph-mon[25866]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:26.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:26 smithi203 ceph-mon[30259]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:26.851 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:26.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:26 smithi193 ceph-mon[30002]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:27.393 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 bash[42918]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-17T08:22:27.394 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 bash[42918]: Writing manifest to image destination 2024-09-17T08:22:27.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:27 smithi047 ceph-mon[25866]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:22:27.722 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-mon[30002]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:22:27.723 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 podman[42918]: 2024-09-17 08:22:27.464651125 +0000 UTC m=+2.179315030 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-17T08:22:27.723 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 podman[42918]: 2024-09-17 08:22:27.538032309 +0000 UTC m=+2.252696201 container create e6d4ab6a3ea3b97fc2bdf3f7e3e52bcc28e9c39ded30563e0f743f4c8f9da732 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T08:22:27.723 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 podman[42918]: 2024-09-17 08:22:27.723203358 +0000 UTC m=+2.437867258 container init e6d4ab6a3ea3b97fc2bdf3f7e3e52bcc28e9c39ded30563e0f743f4c8f9da732 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T08:22:27.784 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:27 smithi203 ceph-mon[30259]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:22:28.144 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 podman[42918]: 2024-09-17 08:22:27.728358495 +0000 UTC m=+2.443022390 container start e6d4ab6a3ea3b97fc2bdf3f7e3e52bcc28e9c39ded30563e0f743f4c8f9da732 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T08:22:28.145 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.732Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-17T08:22:28.145 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.732Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-17T08:22:28.145 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.732Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-17T08:22:28.145 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.732Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-17T08:22:28.145 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.732Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-17T08:22:28.145 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.732Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-17T08:22:28.145 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-17T08:22:28.145 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=arp 2024-09-17T08:22:28.145 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-17T08:22:28.145 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-17T08:22:28.145 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-17T08:22:28.145 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-17T08:22:28.145 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-17T08:22:28.146 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-17T08:22:28.146 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-17T08:22:28.146 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-17T08:22:28.146 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=edac 2024-09-17T08:22:28.146 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-17T08:22:28.147 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-17T08:22:28.147 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-17T08:22:28.147 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-17T08:22:28.147 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-17T08:22:28.147 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-17T08:22:28.147 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-17T08:22:28.147 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-17T08:22:28.147 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-17T08:22:28.147 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-17T08:22:28.147 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-17T08:22:28.147 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-17T08:22:28.147 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-17T08:22:28.147 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-17T08:22:28.147 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-17T08:22:28.148 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-17T08:22:28.148 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=os 2024-09-17T08:22:28.148 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-17T08:22:28.148 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-17T08:22:28.148 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-17T08:22:28.148 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-17T08:22:28.148 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-17T08:22:28.148 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-17T08:22:28.148 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-17T08:22:28.148 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=stat 2024-09-17T08:22:28.148 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-17T08:22:28.148 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-17T08:22:28.148 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-17T08:22:28.148 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=time 2024-09-17T08:22:28.148 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-17T08:22:28.149 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=uname 2024-09-17T08:22:28.149 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-17T08:22:28.149 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-17T08:22:28.149 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-17T08:22:28.149 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-17T08:22:28.149 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b[42993]: ts=2024-09-17T08:22:27.733Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-17T08:22:28.149 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 bash[42918]: e6d4ab6a3ea3b97fc2bdf3f7e3e52bcc28e9c39ded30563e0f743f4c8f9da732 2024-09-17T08:22:28.149 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:22:27 smithi193 systemd[1]: Started Ceph node-exporter.b for 3a66518a-74cd-11ef-bceb-c7b262605968. 2024-09-17T08:22:28.667 INFO:teuthology.orchestra.run.smithi047.stdout:service_type: alertmanager 2024-09-17T08:22:28.668 INFO:teuthology.orchestra.run.smithi047.stdout:service_name: alertmanager 2024-09-17T08:22:28.668 INFO:teuthology.orchestra.run.smithi047.stdout:placement: 2024-09-17T08:22:28.668 INFO:teuthology.orchestra.run.smithi047.stdout: count: 1 2024-09-17T08:22:28.668 INFO:teuthology.orchestra.run.smithi047.stdout: hosts: 2024-09-17T08:22:28.668 INFO:teuthology.orchestra.run.smithi047.stdout: - smithi047=a 2024-09-17T08:22:28.668 INFO:teuthology.orchestra.run.smithi047.stdout:status: 2024-09-17T08:22:28.668 INFO:teuthology.orchestra.run.smithi047.stdout: created: '2024-09-17T08:21:22.629717Z' 2024-09-17T08:22:28.668 INFO:teuthology.orchestra.run.smithi047.stdout: ports: 2024-09-17T08:22:28.668 INFO:teuthology.orchestra.run.smithi047.stdout: - 9093 2024-09-17T08:22:28.668 INFO:teuthology.orchestra.run.smithi047.stdout: - 9094 2024-09-17T08:22:28.668 INFO:teuthology.orchestra.run.smithi047.stdout: running: 0 2024-09-17T08:22:28.669 INFO:teuthology.orchestra.run.smithi047.stdout: size: 1 2024-09-17T08:22:28.669 INFO:teuthology.orchestra.run.smithi047.stdout:events: 2024-09-17T08:22:28.669 INFO:teuthology.orchestra.run.smithi047.stdout:- 2024-09-17T08:22:07.333596Z service:alertmanager [INFO] "service was created" 2024-09-17T08:22:28.669 INFO:teuthology.orchestra.run.smithi047.stdout:--- 2024-09-17T08:22:28.669 INFO:teuthology.orchestra.run.smithi047.stdout:service_type: grafana 2024-09-17T08:22:28.669 INFO:teuthology.orchestra.run.smithi047.stdout:service_name: grafana 2024-09-17T08:22:28.669 INFO:teuthology.orchestra.run.smithi047.stdout:placement: 2024-09-17T08:22:28.669 INFO:teuthology.orchestra.run.smithi047.stdout: count: 1 2024-09-17T08:22:28.669 INFO:teuthology.orchestra.run.smithi047.stdout: hosts: 2024-09-17T08:22:28.669 INFO:teuthology.orchestra.run.smithi047.stdout: - smithi193=a 2024-09-17T08:22:28.669 INFO:teuthology.orchestra.run.smithi047.stdout:spec: 2024-09-17T08:22:28.669 INFO:teuthology.orchestra.run.smithi047.stdout: anonymous_access: true 2024-09-17T08:22:28.669 INFO:teuthology.orchestra.run.smithi047.stdout: protocol: https 2024-09-17T08:22:28.669 INFO:teuthology.orchestra.run.smithi047.stdout:status: 2024-09-17T08:22:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: created: '2024-09-17T08:21:25.116194Z' 2024-09-17T08:22:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: ports: 2024-09-17T08:22:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: - 3000 2024-09-17T08:22:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: running: 0 2024-09-17T08:22:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: size: 1 2024-09-17T08:22:28.670 INFO:teuthology.orchestra.run.smithi047.stdout:events: 2024-09-17T08:22:28.670 INFO:teuthology.orchestra.run.smithi047.stdout:- 2024-09-17T08:22:19.446484Z service:grafana [INFO] "service was created" 2024-09-17T08:22:28.670 INFO:teuthology.orchestra.run.smithi047.stdout:--- 2024-09-17T08:22:28.670 INFO:teuthology.orchestra.run.smithi047.stdout:service_type: mgr 2024-09-17T08:22:28.670 INFO:teuthology.orchestra.run.smithi047.stdout:service_name: mgr 2024-09-17T08:22:28.670 INFO:teuthology.orchestra.run.smithi047.stdout:placement: 2024-09-17T08:22:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: count: 3 2024-09-17T08:22:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: hosts: 2024-09-17T08:22:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: - smithi047=y 2024-09-17T08:22:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: - smithi193=x 2024-09-17T08:22:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: - smithi203=z 2024-09-17T08:22:28.671 INFO:teuthology.orchestra.run.smithi047.stdout:status: 2024-09-17T08:22:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: created: '2024-09-17T08:19:49.835276Z' 2024-09-17T08:22:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: last_refresh: '2024-09-17T08:21:46.564903Z' 2024-09-17T08:22:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: running: 3 2024-09-17T08:22:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: size: 3 2024-09-17T08:22:28.671 INFO:teuthology.orchestra.run.smithi047.stdout:--- 2024-09-17T08:22:28.671 INFO:teuthology.orchestra.run.smithi047.stdout:service_type: mon 2024-09-17T08:22:28.671 INFO:teuthology.orchestra.run.smithi047.stdout:service_name: mon 2024-09-17T08:22:28.671 INFO:teuthology.orchestra.run.smithi047.stdout:placement: 2024-09-17T08:22:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: count: 3 2024-09-17T08:22:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: hosts: 2024-09-17T08:22:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: - smithi047:172.21.15.47=a 2024-09-17T08:22:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: - smithi193:172.21.15.193=b 2024-09-17T08:22:28.672 INFO:teuthology.orchestra.run.smithi047.stdout: - smithi203:172.21.15.203=c 2024-09-17T08:22:28.672 INFO:teuthology.orchestra.run.smithi047.stdout:status: 2024-09-17T08:22:28.672 INFO:teuthology.orchestra.run.smithi047.stdout: created: '2024-09-17T08:19:24.426052Z' 2024-09-17T08:22:28.672 INFO:teuthology.orchestra.run.smithi047.stdout: last_refresh: '2024-09-17T08:21:46.564816Z' 2024-09-17T08:22:28.672 INFO:teuthology.orchestra.run.smithi047.stdout: running: 3 2024-09-17T08:22:28.672 INFO:teuthology.orchestra.run.smithi047.stdout: size: 3 2024-09-17T08:22:28.672 INFO:teuthology.orchestra.run.smithi047.stdout:--- 2024-09-17T08:22:28.672 INFO:teuthology.orchestra.run.smithi047.stdout:service_type: node-exporter 2024-09-17T08:22:28.672 INFO:teuthology.orchestra.run.smithi047.stdout:service_name: node-exporter 2024-09-17T08:22:28.672 INFO:teuthology.orchestra.run.smithi047.stdout:placement: 2024-09-17T08:22:28.672 INFO:teuthology.orchestra.run.smithi047.stdout: count: 3 2024-09-17T08:22:28.672 INFO:teuthology.orchestra.run.smithi047.stdout: hosts: 2024-09-17T08:22:28.672 INFO:teuthology.orchestra.run.smithi047.stdout: - smithi047=a 2024-09-17T08:22:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: - smithi193=b 2024-09-17T08:22:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: - smithi203=c 2024-09-17T08:22:28.673 INFO:teuthology.orchestra.run.smithi047.stdout:status: 2024-09-17T08:22:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: created: '2024-09-17T08:21:20.187923Z' 2024-09-17T08:22:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: ports: 2024-09-17T08:22:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: - 9100 2024-09-17T08:22:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: running: 0 2024-09-17T08:22:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: size: 3 2024-09-17T08:22:28.673 INFO:teuthology.orchestra.run.smithi047.stdout:events: 2024-09-17T08:22:28.673 INFO:teuthology.orchestra.run.smithi047.stdout:- 2024-09-17T08:22:27.883144Z service:node-exporter [INFO] "service was created" 2024-09-17T08:22:28.673 INFO:teuthology.orchestra.run.smithi047.stdout:--- 2024-09-17T08:22:28.673 INFO:teuthology.orchestra.run.smithi047.stdout:service_type: osd 2024-09-17T08:22:28.673 INFO:teuthology.orchestra.run.smithi047.stdout:service_name: osd 2024-09-17T08:22:28.673 INFO:teuthology.orchestra.run.smithi047.stdout:unmanaged: true 2024-09-17T08:22:28.673 INFO:teuthology.orchestra.run.smithi047.stdout:spec: 2024-09-17T08:22:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: filter_logic: AND 2024-09-17T08:22:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: objectstore: bluestore 2024-09-17T08:22:28.674 INFO:teuthology.orchestra.run.smithi047.stdout:status: 2024-09-17T08:22:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: container_image_id: 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 2024-09-17T08:22:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: container_image_name: quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:22:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: last_refresh: '2024-09-17T08:21:46.564982Z' 2024-09-17T08:22:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: running: 3 2024-09-17T08:22:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: size: 3 2024-09-17T08:22:28.674 INFO:teuthology.orchestra.run.smithi047.stdout:--- 2024-09-17T08:22:28.674 INFO:teuthology.orchestra.run.smithi047.stdout:service_type: prometheus 2024-09-17T08:22:28.674 INFO:teuthology.orchestra.run.smithi047.stdout:service_name: prometheus 2024-09-17T08:22:28.674 INFO:teuthology.orchestra.run.smithi047.stdout:placement: 2024-09-17T08:22:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: count: 1 2024-09-17T08:22:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: hosts: 2024-09-17T08:22:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: - smithi193=a 2024-09-17T08:22:28.675 INFO:teuthology.orchestra.run.smithi047.stdout:status: 2024-09-17T08:22:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: created: '2024-09-17T08:21:17.690565Z' 2024-09-17T08:22:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: last_refresh: '2024-09-17T08:21:47.347241Z' 2024-09-17T08:22:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: ports: 2024-09-17T08:22:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: - 9095 2024-09-17T08:22:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: running: 1 2024-09-17T08:22:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: size: 1 2024-09-17T08:22:28.675 INFO:teuthology.orchestra.run.smithi047.stdout:--- 2024-09-17T08:22:28.675 INFO:teuthology.orchestra.run.smithi047.stdout:service_type: rgw 2024-09-17T08:22:28.675 INFO:teuthology.orchestra.run.smithi047.stdout:service_id: foo.a 2024-09-17T08:22:28.675 INFO:teuthology.orchestra.run.smithi047.stdout:service_name: rgw.foo.a 2024-09-17T08:22:28.675 INFO:teuthology.orchestra.run.smithi047.stdout:placement: 2024-09-17T08:22:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: count: 1 2024-09-17T08:22:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: hosts: 2024-09-17T08:22:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: - smithi047=foo.a 2024-09-17T08:22:28.676 INFO:teuthology.orchestra.run.smithi047.stdout:status: 2024-09-17T08:22:28.676 INFO:teuthology.orchestra.run.smithi047.stdout: created: '2024-09-17T08:21:18.345113Z' 2024-09-17T08:22:28.676 INFO:teuthology.orchestra.run.smithi047.stdout: last_refresh: '2024-09-17T08:21:52.493748Z' 2024-09-17T08:22:28.676 INFO:teuthology.orchestra.run.smithi047.stdout: ports: 2024-09-17T08:22:28.676 INFO:teuthology.orchestra.run.smithi047.stdout: - 80 2024-09-17T08:22:28.676 INFO:teuthology.orchestra.run.smithi047.stdout: running: 1 2024-09-17T08:22:28.676 INFO:teuthology.orchestra.run.smithi047.stdout: size: 1 2024-09-17T08:22:28.872 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:28 smithi203 bash[43795]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-17T08:22:28.996 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:28 smithi047 ceph-mon[25866]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:28.997 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:28 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:28.997 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:28 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:28.997 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:28 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:28.997 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:28 smithi047 ceph-mon[25866]: Deploying daemon node-exporter.c on smithi203 2024-09-17T08:22:29.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:28 smithi193 ceph-mon[30002]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:29.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:28 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:29.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:28 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:29.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:28 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:29.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:28 smithi193 ceph-mon[30002]: Deploying daemon node-exporter.c on smithi203 2024-09-17T08:22:29.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:28 smithi203 ceph-mon[30259]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:29.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:28 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:29.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:28 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:29.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:28 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:29.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:28 smithi203 ceph-mon[30259]: Deploying daemon node-exporter.c on smithi203 2024-09-17T08:22:29.285 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:29 smithi203 bash[43795]: Getting image source signatures 2024-09-17T08:22:29.285 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:29 smithi203 bash[43795]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-17T08:22:29.285 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:29 smithi203 bash[43795]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-17T08:22:29.285 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:29 smithi203 bash[43795]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-17T08:22:29.334 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls | grep '"'"'^osd '"'"'' 2024-09-17T08:22:29.577 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:29.988 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:29 smithi047 ceph-mon[25866]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-17T08:22:30.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:29 smithi193 ceph-mon[30002]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-17T08:22:30.284 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:29 smithi203 ceph-mon[30259]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-17T08:22:30.784 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:30 smithi203 bash[43795]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-17T08:22:30.784 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:30 smithi203 bash[43795]: Writing manifest to image destination 2024-09-17T08:22:30.997 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:30 smithi047 ceph-mon[25866]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:30.997 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:30 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:22:31.143 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:30 smithi193 ceph-mon[30002]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:31.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:30 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:22:31.213 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:30 smithi203 ceph-mon[30259]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:31.213 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:30 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:22:31.214 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:30 smithi203 podman[43795]: 2024-09-17 08:22:30.896404922 +0000 UTC m=+2.202315824 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-17T08:22:31.214 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:30 smithi203 podman[43795]: 2024-09-17 08:22:30.99495546 +0000 UTC m=+2.300866349 container create a37d99d9c2e4209eb042c205d16cf3f47798de25580fde8890f6b1f817d1f4e9 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c, maintainer=The Prometheus Authors ) 2024-09-17T08:22:31.214 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 podman[43795]: 2024-09-17 08:22:31.2136107 +0000 UTC m=+2.519521598 container init a37d99d9c2e4209eb042c205d16cf3f47798de25580fde8890f6b1f817d1f4e9 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c, maintainer=The Prometheus Authors ) 2024-09-17T08:22:31.417 INFO:teuthology.orchestra.run.smithi047.stdout:osd 3 44s ago - 2024-09-17T08:22:31.534 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 podman[43795]: 2024-09-17 08:22:31.218898436 +0000 UTC m=+2.524809337 container start a37d99d9c2e4209eb042c205d16cf3f47798de25580fde8890f6b1f817d1f4e9 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c, maintainer=The Prometheus Authors ) 2024-09-17T08:22:31.534 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.222Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-17T08:22:31.535 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.222Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-17T08:22:31.535 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.223Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-17T08:22:31.535 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.223Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-17T08:22:31.535 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.223Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-17T08:22:31.535 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-17T08:22:31.535 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-17T08:22:31.535 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=arp 2024-09-17T08:22:31.535 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-17T08:22:31.535 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-17T08:22:31.535 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-17T08:22:31.535 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-17T08:22:31.535 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-17T08:22:31.535 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-17T08:22:31.536 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-17T08:22:31.536 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-17T08:22:31.536 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=edac 2024-09-17T08:22:31.536 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-17T08:22:31.536 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-17T08:22:31.536 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-17T08:22:31.536 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-17T08:22:31.536 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-17T08:22:31.537 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-17T08:22:31.537 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-17T08:22:31.537 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-17T08:22:31.537 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-17T08:22:31.537 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-17T08:22:31.537 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-17T08:22:31.537 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-17T08:22:31.537 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-17T08:22:31.537 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-17T08:22:31.537 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-17T08:22:31.537 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-17T08:22:31.537 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=os 2024-09-17T08:22:31.537 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-17T08:22:31.537 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-17T08:22:31.538 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-17T08:22:31.538 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-17T08:22:31.538 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-17T08:22:31.538 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-17T08:22:31.538 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-17T08:22:31.538 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=stat 2024-09-17T08:22:31.538 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-17T08:22:31.538 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-17T08:22:31.538 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-17T08:22:31.538 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=time 2024-09-17T08:22:31.538 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-17T08:22:31.538 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=uname 2024-09-17T08:22:31.538 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-17T08:22:31.538 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-17T08:22:31.539 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.224Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-17T08:22:31.539 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.225Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-17T08:22:31.539 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c[43869]: ts=2024-09-17T08:22:31.225Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-17T08:22:31.539 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 bash[43795]: a37d99d9c2e4209eb042c205d16cf3f47798de25580fde8890f6b1f817d1f4e9 2024-09-17T08:22:31.539 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:22:31 smithi203 systemd[1]: Started Ceph node-exporter.c for 3a66518a-74cd-11ef-bceb-c7b262605968. 2024-09-17T08:22:32.086 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-17T08:22:32.099 INFO:tasks.cephadm:Teardown begin 2024-09-17T08:22:32.100 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:22:32.128 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:22:32.157 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:22:32.185 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-17T08:22:32.185 DEBUG:teuthology.orchestra.run.smithi047:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-17T08:22:32.202 DEBUG:teuthology.orchestra.run.smithi193:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-17T08:22:32.217 DEBUG:teuthology.orchestra.run.smithi203:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-17T08:22:32.240 INFO:tasks.cephadm:Stopping all daemons... 2024-09-17T08:22:32.240 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-17T08:22:32.241 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mon.a 2024-09-17T08:22:32.355 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:32 smithi047 systemd[1]: Stopping Ceph mon.a for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:22:32.608 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:32 smithi047 ceph-mon[25866]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:32.608 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:32 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:32.608 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:32 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:32.608 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:32 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:32.608 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:32 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:32.608 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:32 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:32.608 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:32 smithi047 ceph-mon[25866]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:22:32.608 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:32 smithi047 ceph-mon[25866]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:22:32.643 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:32 smithi193 ceph-mon[30002]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:32.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:32 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:32.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:32 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:32.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:32 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:32.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:32 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:32.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:32 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:32.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:32 smithi193 ceph-mon[30002]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:22:32.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:32 smithi193 ceph-mon[30002]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:22:32.777 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:32 smithi203 ceph-mon[30259]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:22:32.777 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:32 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:32.777 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:32 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:32.777 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:32 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:32.777 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:32 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:32.777 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:32 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' 2024-09-17T08:22:32.777 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:32 smithi203 ceph-mon[30259]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:22:32.777 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:32 smithi203 ceph-mon[30259]: from='mgr.14421 172.21.15.47:0/2969563414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:22:33.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:33 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-a[25842]: 2024-09-17T08:22:33.006+0000 7f7f77797640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:22:33.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:33 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-a[25842]: 2024-09-17T08:22:33.006+0000 7f7f77797640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-17T08:22:33.290 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:22:33 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: [17/Sep/2024:08:22:33] ENGINE Bus STOPPING 2024-09-17T08:22:33.291 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:22:33 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: [17/Sep/2024:08:22:33] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T08:22:33.291 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:22:33 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: [17/Sep/2024:08:22:33] ENGINE Bus STOPPED 2024-09-17T08:22:33.291 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:22:33 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: [17/Sep/2024:08:22:33] ENGINE Bus STARTING 2024-09-17T08:22:33.291 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:22:33 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: [17/Sep/2024:08:22:33] ENGINE Serving on http://:::9283 2024-09-17T08:22:33.291 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:22:33 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: [17/Sep/2024:08:22:33] ENGINE Bus STARTED 2024-09-17T08:22:33.674 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:33 smithi047 podman[58436]: 2024-09-17 08:22:33.398985184 +0000 UTC m=+0.474983323 container died a0d16c90f0857baa4dd2177ab84ed678902f3e968a053395d6b5394c8041c0a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-a, RELEASE=reef-19e2a2b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-17T08:22:34.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:33 smithi047 podman[58436]: 2024-09-17 08:22:33.781699574 +0000 UTC m=+0.857697708 container cleanup a0d16c90f0857baa4dd2177ab84ed678902f3e968a053395d6b5394c8041c0a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-17T08:22:34.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:33 smithi047 bash[58436]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-a 2024-09-17T08:22:34.762 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 17 08:22:34 smithi047 podman[58543]: 2024-09-17 08:22:34.417735514 +0000 UTC m=+1.242314565 container remove a0d16c90f0857baa4dd2177ab84ed678902f3e968a053395d6b5394c8041c0a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:22:34.763 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:22:34 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: ::ffff:172.21.15.193 - - [17/Sep/2024:08:22:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:22:34.786 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mon.a.service' 2024-09-17T08:22:34.858 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:22:34.858 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-17T08:22:34.858 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-17T08:22:34.858 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mon.b 2024-09-17T08:22:35.170 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:34 smithi193 systemd[1]: Stopping Ceph mon.b for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:22:35.643 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:35 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-b[29979]: 2024-09-17T08:22:35.175+0000 7fc86b4eb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:22:35.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:35 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-b[29979]: 2024-09-17T08:22:35.175+0000 7fc86b4eb640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-17T08:22:35.644 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:35 smithi193 podman[43611]: 2024-09-17 08:22:35.420151427 +0000 UTC m=+0.353919442 container died cc6a76045055bd58b90ab0b1a0dfe3bab17bb8ce1f851ef58483152306fac624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD) 2024-09-17T08:22:36.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:35 smithi193 podman[43611]: 2024-09-17 08:22:35.789694963 +0000 UTC m=+0.723462967 container cleanup cc6a76045055bd58b90ab0b1a0dfe3bab17bb8ce1f851ef58483152306fac624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-17T08:22:36.144 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 17 08:22:35 smithi193 bash[43611]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-b 2024-09-17T08:22:36.482 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mon.b.service' 2024-09-17T08:22:36.527 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:22:36.527 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-17T08:22:36.527 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-17T08:22:36.528 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mon.c 2024-09-17T08:22:36.884 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:36 smithi203 systemd[1]: Stopping Ceph mon.c for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:22:37.198 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:36 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-c[30222]: 2024-09-17T08:22:36.882+0000 7f7d6f657640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:22:37.198 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:36 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-c[30222]: 2024-09-17T08:22:36.882+0000 7f7d6f657640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-17T08:22:37.198 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:36 smithi203 podman[44315]: 2024-09-17 08:22:36.945573635 +0000 UTC m=+0.216916866 container died 66cd9b73a652108bffa0e6361733a90223127ea43bcd309d18368fdf1848d995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-c, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2) 2024-09-17T08:22:37.457 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:37 smithi203 podman[44315]: 2024-09-17 08:22:37.289421436 +0000 UTC m=+0.560764668 container cleanup 66cd9b73a652108bffa0e6361733a90223127ea43bcd309d18368fdf1848d995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-17T08:22:37.458 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:37 smithi203 bash[44315]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-c 2024-09-17T08:22:37.709 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 17 08:22:37 smithi203 podman[44331]: 2024-09-17 08:22:37.45818768 +0000 UTC m=+0.508231407 container remove 66cd9b73a652108bffa0e6361733a90223127ea43bcd309d18368fdf1848d995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-17T08:22:37.714 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mon.c.service' 2024-09-17T08:22:37.759 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:22:37.759 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-17T08:22:37.759 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-17T08:22:37.759 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mgr.y 2024-09-17T08:22:38.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:22:37 smithi047 systemd[1]: Stopping Ceph mgr.y for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:22:38.366 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:22:38 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y[26120]: 2024-09-17T08:22:38.041+0000 7f191e9d8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:22:38.367 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:22:38 smithi047 podman[58786]: 2024-09-17 08:22:38.073538342 +0000 UTC m=+0.114986921 container died e29578f4bd1a84766aa2430f114d2b19e323ea3ab1e39c2ca33f63e62a080ed9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-09-17T08:22:38.716 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:22:38 smithi047 podman[58786]: 2024-09-17 08:22:38.448136457 +0000 UTC m=+0.489585029 container cleanup e29578f4bd1a84766aa2430f114d2b19e323ea3ab1e39c2ca33f63e62a080ed9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:22:38.716 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 17 08:22:38 smithi047 bash[58786]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-y 2024-09-17T08:22:38.878 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-17T08:22:38.915 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:22:38.915 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-17T08:22:38.915 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-17T08:22:38.915 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mgr.x 2024-09-17T08:22:39.143 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:22:38 smithi193 systemd[1]: Stopping Ceph mgr.x for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:22:39.144 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:22:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:22:38.860Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.47:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.47:8765: connect: connection refused" 2024-09-17T08:22:39.144 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:22:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:22:38.860Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.47:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.47:8765: connect: connection refused" 2024-09-17T08:22:39.144 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:22:38 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:22:38.860Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.47:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.47:8765: connect: connection refused" 2024-09-17T08:22:39.572 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:22:39 smithi193 podman[43755]: 2024-09-17 08:22:39.269919059 +0000 UTC m=+0.163045598 container died d54a413ea2d5429f20bfe770fe1bdf29b72f49e2d23f3b8e4df24806e6e4721b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:22:39.872 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:22:39 smithi193 podman[43755]: 2024-09-17 08:22:39.713302964 +0000 UTC m=+0.606429539 container cleanup d54a413ea2d5429f20bfe770fe1bdf29b72f49e2d23f3b8e4df24806e6e4721b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-17T08:22:39.872 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 17 08:22:39 smithi193 bash[43755]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-x 2024-09-17T08:22:40.087 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-17T08:22:40.120 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:22:40.121 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-17T08:22:40.121 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-17T08:22:40.121 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mgr.z 2024-09-17T08:22:40.469 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:22:40 smithi203 systemd[1]: Stopping Ceph mgr.z for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:22:40.770 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:22:40 smithi203 podman[44444]: 2024-09-17 08:22:40.469206907 +0000 UTC m=+0.154279545 container died 66a90b7fef045b4762492f6183857ebf318307bcb2e641ec716bd0bccdb9b4c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-17T08:22:41.034 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:22:40 smithi203 podman[44444]: 2024-09-17 08:22:40.770680314 +0000 UTC m=+0.455752961 container cleanup 66a90b7fef045b4762492f6183857ebf318307bcb2e641ec716bd0bccdb9b4c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-17T08:22:41.034 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:22:40 smithi203 bash[44444]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z 2024-09-17T08:22:41.034 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:22:40 smithi203 podman[44461]: 2024-09-17 08:22:40.997019627 +0000 UTC m=+0.517936447 container remove 66a90b7fef045b4762492f6183857ebf318307bcb2e641ec716bd0bccdb9b4c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-mgr-z, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=) 2024-09-17T08:22:41.034 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 17 08:22:40 smithi203 systemd[1]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-17T08:22:41.153 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-17T08:22:41.186 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:22:41.186 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-17T08:22:41.186 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-17T08:22:41.187 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-3a66518a-74cd-11ef-bceb-c7b262605968@osd.0 2024-09-17T08:22:41.479 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:22:41 smithi047 systemd[1]: Stopping Ceph osd.0 for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:22:41.789 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:22:41 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-0[44053]: 2024-09-17T08:22:41.478+0000 7faefddc9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:22:41.789 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:22:41 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-0[44053]: 2024-09-17T08:22:41.478+0000 7faefddc9640 -1 osd.0 31 *** Got signal Terminated *** 2024-09-17T08:22:41.789 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:22:41 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-0[44053]: 2024-09-17T08:22:41.478+0000 7faefddc9640 -1 osd.0 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T08:22:47.161 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:22:46 smithi047 podman[58916]: 2024-09-17 08:22:46.88429632 +0000 UTC m=+5.504366737 container died 34bb02166ca69cc4379867ff74e66da67bb2b7bc296ea4a914218213912d85ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-17T08:22:47.528 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:22:47 smithi047 podman[58916]: 2024-09-17 08:22:47.335001091 +0000 UTC m=+5.955071507 container cleanup 34bb02166ca69cc4379867ff74e66da67bb2b7bc296ea4a914218213912d85ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-0, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-17T08:22:47.528 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:22:47 smithi047 bash[58916]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-0 2024-09-17T08:22:47.789 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:22:47 smithi047 podman[58940]: 2024-09-17 08:22:47.527870855 +0000 UTC m=+0.633277381 container remove 34bb02166ca69cc4379867ff74e66da67bb2b7bc296ea4a914218213912d85ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:22:48.280 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:22:47 smithi047 podman[59032]: 2024-09-17 08:22:47.854535493 +0000 UTC m=+0.016196852 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:22:48.280 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:22:47 smithi047 podman[59032]: 2024-09-17 08:22:47.97039307 +0000 UTC m=+0.132054409 container create 5c619d7e3d1bd2ea49787c8819092cd8c0e6398ce0d6938006930201a1c0e090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T08:22:48.280 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:22:48 smithi047 podman[59032]: 2024-09-17 08:22:48.189357832 +0000 UTC m=+0.351019191 container init 5c619d7e3d1bd2ea49787c8819092cd8c0e6398ce0d6938006930201a1c0e090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-17T08:22:48.281 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:22:48 smithi047 podman[59032]: 2024-09-17 08:22:48.194951056 +0000 UTC m=+0.356612403 container start 5c619d7e3d1bd2ea49787c8819092cd8c0e6398ce0d6938006930201a1c0e090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:22:48.281 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:22:48 smithi047 podman[59032]: 2024-09-17 08:22:48.280741107 +0000 UTC m=+0.442402456 container attach 5c619d7e3d1bd2ea49787c8819092cd8c0e6398ce0d6938006930201a1c0e090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, ceph=True, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-17T08:22:48.539 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 17 08:22:48 smithi047 podman[59032]: 2024-09-17 08:22:48.385288376 +0000 UTC m=+0.546949723 container died 5c619d7e3d1bd2ea49787c8819092cd8c0e6398ce0d6938006930201a1c0e090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-17T08:22:49.209 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@osd.0.service' 2024-09-17T08:22:49.254 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:22:49.254 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-17T08:22:49.254 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-17T08:22:49.254 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-3a66518a-74cd-11ef-bceb-c7b262605968@osd.1 2024-09-17T08:22:49.558 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:22:49 smithi193 systemd[1]: Stopping Ceph osd.1 for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:22:49.893 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:22:49 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-1[34409]: 2024-09-17T08:22:49.558+0000 7f573ec0d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:22:49.893 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:22:49 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-1[34409]: 2024-09-17T08:22:49.558+0000 7f573ec0d640 -1 osd.1 31 *** Got signal Terminated *** 2024-09-17T08:22:49.894 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:22:49 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-1[34409]: 2024-09-17T08:22:49.558+0000 7f573ec0d640 -1 osd.1 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T08:22:55.282 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:22:55 smithi193 podman[43887]: 2024-09-17 08:22:55.015094958 +0000 UTC m=+5.568018629 container died d0e3c56000715950f2738e764bc4da3e247172eaa2c50e4370c838356e2f448d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-1, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2) 2024-09-17T08:22:55.640 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:22:55 smithi193 podman[43887]: 2024-09-17 08:22:55.372952565 +0000 UTC m=+5.925876228 container cleanup d0e3c56000715950f2738e764bc4da3e247172eaa2c50e4370c838356e2f448d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-1, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:22:55.640 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:22:55 smithi193 bash[43887]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-1 2024-09-17T08:22:55.893 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:22:55 smithi193 podman[43907]: 2024-09-17 08:22:55.640598538 +0000 UTC m=+0.619727781 container remove d0e3c56000715950f2738e764bc4da3e247172eaa2c50e4370c838356e2f448d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731) 2024-09-17T08:22:56.226 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:22:55 smithi193 podman[44002]: 2024-09-17 08:22:55.966440528 +0000 UTC m=+0.094901602 container create 30f1f3148bab1f9ae95b7cd4de06a3491c41bdd549bd69a20e4e79f536c349e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-17T08:22:56.227 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:22:55 smithi193 podman[44002]: 2024-09-17 08:22:55.88730891 +0000 UTC m=+0.015769954 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:22:56.227 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:22:56 smithi193 podman[44002]: 2024-09-17 08:22:56.160184879 +0000 UTC m=+0.288645912 container init 30f1f3148bab1f9ae95b7cd4de06a3491c41bdd549bd69a20e4e79f536c349e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-17T08:22:56.227 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:22:56 smithi193 podman[44002]: 2024-09-17 08:22:56.164783899 +0000 UTC m=+0.293244937 container start 30f1f3148bab1f9ae95b7cd4de06a3491c41bdd549bd69a20e4e79f536c349e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-17T08:22:56.227 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:22:56 smithi193 podman[44002]: 2024-09-17 08:22:56.226656565 +0000 UTC m=+0.355117613 container attach 30f1f3148bab1f9ae95b7cd4de06a3491c41bdd549bd69a20e4e79f536c349e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True) 2024-09-17T08:22:56.644 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:22:56 smithi193 podman[44002]: 2024-09-17 08:22:56.355109798 +0000 UTC m=+0.483570848 container died 30f1f3148bab1f9ae95b7cd4de06a3491c41bdd549bd69a20e4e79f536c349e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T08:22:56.901 INFO:journalctl@ceph.osd.1.smithi193.stdout:Sep 17 08:22:56 smithi193 podman[44046]: 2024-09-17 08:22:56.888014634 +0000 UTC m=+0.527197927 container remove 30f1f3148bab1f9ae95b7cd4de06a3491c41bdd549bd69a20e4e79f536c349e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-17T08:22:56.906 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@osd.1.service' 2024-09-17T08:22:56.943 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:22:56.943 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-17T08:22:56.944 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-17T08:22:56.944 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-3a66518a-74cd-11ef-bceb-c7b262605968@osd.2 2024-09-17T08:22:57.251 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:22:56 smithi203 systemd[1]: Stopping Ceph osd.2 for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:22:57.534 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:22:57 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-2[35366]: 2024-09-17T08:22:57.249+0000 7f168a0f1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:22:57.534 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:22:57 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-2[35366]: 2024-09-17T08:22:57.249+0000 7f168a0f1640 -1 osd.2 31 *** Got signal Terminated *** 2024-09-17T08:22:57.534 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:22:57 smithi203 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-2[35366]: 2024-09-17T08:22:57.249+0000 7f168a0f1640 -1 osd.2 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T08:23:03.034 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:23:02 smithi203 podman[44577]: 2024-09-17 08:23:02.72809827 +0000 UTC m=+5.590289078 container stop 7dbf3d9cd87f53b2f84606314bdf7535cf8500902d95d6549b8b5d54485b0e50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-2, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True) 2024-09-17T08:23:03.034 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:23:02 smithi203 podman[44577]: 2024-09-17 08:23:02.754240511 +0000 UTC m=+5.616431335 container died 7dbf3d9cd87f53b2f84606314bdf7535cf8500902d95d6549b8b5d54485b0e50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:23:03.415 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:23:03 smithi203 podman[44577]: 2024-09-17 08:23:03.272993497 +0000 UTC m=+6.135184298 container cleanup 7dbf3d9cd87f53b2f84606314bdf7535cf8500902d95d6549b8b5d54485b0e50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:23:03.416 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:23:03 smithi203 bash[44577]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-2 2024-09-17T08:23:03.784 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:23:03 smithi203 podman[44610]: 2024-09-17 08:23:03.415921691 +0000 UTC m=+0.679358362 container remove 7dbf3d9cd87f53b2f84606314bdf7535cf8500902d95d6549b8b5d54485b0e50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-2, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-17T08:23:04.085 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:23:03 smithi203 podman[44706]: 2024-09-17 08:23:03.800127429 +0000 UTC m=+0.084137409 container create eaedab56db494425a8b0d459df05fbfeaf4856f69d1944e19525bf0030321b40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T08:23:04.085 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:23:03 smithi203 podman[44706]: 2024-09-17 08:23:03.732017859 +0000 UTC m=+0.016027802 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:23:04.085 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:23:03 smithi203 podman[44706]: 2024-09-17 08:23:03.994105166 +0000 UTC m=+0.278115116 container init eaedab56db494425a8b0d459df05fbfeaf4856f69d1944e19525bf0030321b40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS) 2024-09-17T08:23:04.085 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:23:03 smithi203 podman[44706]: 2024-09-17 08:23:03.999470268 +0000 UTC m=+0.283480213 container start eaedab56db494425a8b0d459df05fbfeaf4856f69d1944e19525bf0030321b40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:23:04.085 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:23:04 smithi203 podman[44706]: 2024-09-17 08:23:04.085327569 +0000 UTC m=+0.369337523 container attach eaedab56db494425a8b0d459df05fbfeaf4856f69d1944e19525bf0030321b40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-17T08:23:04.479 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:23:04 smithi203 podman[44706]: 2024-09-17 08:23:04.187065853 +0000 UTC m=+0.471075798 container died eaedab56db494425a8b0d459df05fbfeaf4856f69d1944e19525bf0030321b40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-17T08:23:04.729 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:23:04 smithi203 podman[44749]: 2024-09-17 08:23:04.713137354 +0000 UTC m=+0.517244809 container remove eaedab56db494425a8b0d459df05fbfeaf4856f69d1944e19525bf0030321b40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-17T08:23:04.729 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:23:04 smithi203 systemd[1]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968@osd.2.service: Deactivated successfully. 2024-09-17T08:23:04.729 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 17 08:23:04 smithi203 systemd[1]: Stopped Ceph osd.2 for 3a66518a-74cd-11ef-bceb-c7b262605968. 2024-09-17T08:23:04.732 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@osd.2.service' 2024-09-17T08:23:04.769 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:23:04.770 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-17T08:23:04.770 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-17T08:23:04.770 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-3a66518a-74cd-11ef-bceb-c7b262605968@rgw.foo.a 2024-09-17T08:23:05.072 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 17 08:23:04 smithi047 systemd[1]: Stopping Ceph rgw.foo.a for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:23:05.539 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 17 08:23:05 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-rgw-foo-a[50278]: 2024-09-17T08:23:05.070+0000 7f4cf1460640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:23:05.539 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 17 08:23:05 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-rgw-foo-a[50278]: 2024-09-17T08:23:05.070+0000 7f4cf4ccb880 -1 shutting down 2024-09-17T08:23:15.360 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 17 08:23:15 smithi047 bash[59173]: time="2024-09-17T08:23:15Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-3a66518a-74cd-11ef-bceb-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-17T08:23:15.360 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 17 08:23:15 smithi047 podman[59173]: 2024-09-17 08:23:15.096723128 +0000 UTC m=+10.121060598 container died f02521794540c647642c45178861d87603137270f908470ce9c304feea3a91f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-rgw-foo-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, ceph=True) 2024-09-17T08:23:15.718 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 17 08:23:15 smithi047 podman[59173]: 2024-09-17 08:23:15.525697223 +0000 UTC m=+10.550034692 container cleanup f02521794540c647642c45178861d87603137270f908470ce9c304feea3a91f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-17T08:23:15.718 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 17 08:23:15 smithi047 bash[59173]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968-rgw-foo-a 2024-09-17T08:23:15.963 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@rgw.foo.a.service' 2024-09-17T08:23:15.985 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 17 08:23:15 smithi047 podman[59197]: 2024-09-17 08:23:15.718532728 +0000 UTC m=+0.616110719 container remove f02521794540c647642c45178861d87603137270f908470ce9c304feea3a91f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-rgw-foo-a, ceph=True, GIT_CLEAN=True, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:23:15.985 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 17 08:23:15 smithi047 systemd[1]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968@rgw.foo.a.service: Main process exited, code=exited, status=137/n/a 2024-09-17T08:23:15.986 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 17 08:23:15 smithi047 systemd[1]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968@rgw.foo.a.service: Failed with result 'exit-code'. 2024-09-17T08:23:15.986 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 17 08:23:15 smithi047 systemd[1]: Stopped Ceph rgw.foo.a for 3a66518a-74cd-11ef-bceb-c7b262605968. 2024-09-17T08:23:15.986 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 17 08:23:15 smithi047 systemd[1]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968@rgw.foo.a.service: Consumed 1.343s CPU time. 2024-09-17T08:23:15.999 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:23:15.999 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-17T08:23:16.000 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-17T08:23:16.000 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-3a66518a-74cd-11ef-bceb-c7b262605968@prometheus.a 2024-09-17T08:23:16.315 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:23:16 smithi193 systemd[1]: Stopping Ceph prometheus.a for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:23:16.638 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:23:16 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:23:16.339Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-17T08:23:16.638 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:23:16 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:23:16.339Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-17T08:23:16.639 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:23:16 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:23:16.339Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-17T08:23:16.639 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:23:16 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:23:16.339Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-17T08:23:16.639 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:23:16 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:23:16.339Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-17T08:23:16.639 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:23:16 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:23:16.340Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-17T08:23:16.639 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:23:16 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:23:16.340Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-17T08:23:16.639 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:23:16 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:23:16.340Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-17T08:23:16.639 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:23:16 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:23:16.340Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-17T08:23:16.639 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:23:16 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:23:16.340Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-17T08:23:16.639 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:23:16 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:23:16.340Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-17T08:23:16.639 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:23:16 smithi193 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a[40831]: ts=2024-09-17T08:23:16.340Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-17T08:23:16.639 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:23:16 smithi193 podman[44132]: 2024-09-17 08:23:16.350472039 +0000 UTC m=+0.159764415 container died 1ba33cd85096b236e70eacb887a100da559613e99ee0f4494244e93b9c8dd0a7 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:23:16.896 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:23:16 smithi193 podman[44132]: 2024-09-17 08:23:16.763139572 +0000 UTC m=+0.572431933 container cleanup 1ba33cd85096b236e70eacb887a100da559613e99ee0f4494244e93b9c8dd0a7 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:23:16.896 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 17 08:23:16 smithi193 bash[44132]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968-prometheus-a 2024-09-17T08:23:17.144 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3a66518a-74cd-11ef-bceb-c7b262605968@prometheus.a.service' 2024-09-17T08:23:17.180 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:23:17.180 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-17T08:23:17.180 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-17T08:23:17.339 INFO:teuthology.orchestra.run.smithi047.stdout:Deleting cluster with fsid: 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:23:18.441 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:23:18 smithi047 systemd[1]: Stopping Ceph alertmanager.a for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:23:18.739 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:23:18 smithi047 ceph-3a66518a-74cd-11ef-bceb-c7b262605968-alertmanager-a[55876]: ts=2024-09-17T08:23:18.440Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-17T08:23:18.740 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:23:18 smithi047 podman[59498]: 2024-09-17 08:23:18.471244517 +0000 UTC m=+0.123912967 container died c04cdbc35cb1b475ee1d4c6d6fb4e73d7774e658dac4658f8252779c800e90af (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:23:19.039 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:23:18 smithi047 podman[59498]: 2024-09-17 08:23:18.829948498 +0000 UTC m=+0.482616948 container cleanup c04cdbc35cb1b475ee1d4c6d6fb4e73d7774e658dac4658f8252779c800e90af (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:23:19.040 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:23:18 smithi047 bash[59498]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968-alertmanager-a 2024-09-17T08:23:19.510 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:23:19 smithi047 podman[59518]: 2024-09-17 08:23:19.08142216 +0000 UTC m=+0.613313515 container remove c04cdbc35cb1b475ee1d4c6d6fb4e73d7774e658dac4658f8252779c800e90af (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:23:19.510 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:23:19 smithi047 podman[59518]: 2024-09-17 08:23:19.16518783 +0000 UTC m=+0.697079187 volume remove 6609a94f557af636c7a38787b06fbe40c44094de5d507a25fa329f617b1b0827 2024-09-17T08:23:19.511 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:23:19 smithi047 systemd[1]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-17T08:23:19.511 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 17 08:23:19 smithi047 systemd[1]: Stopped Ceph alertmanager.a for 3a66518a-74cd-11ef-bceb-c7b262605968. 2024-09-17T08:23:19.790 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:23:19 smithi047 systemd[1]: Stopping Ceph node-exporter.a for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:23:19.790 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:23:19 smithi047 podman[59629]: 2024-09-17 08:23:19.738640824 +0000 UTC m=+0.085112086 container died a7d7148fb0b5bb585900ca783c70061e264e98ce4b205b3a6745b1b0eb9c5aba (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T08:23:20.506 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:23:20 smithi047 podman[59629]: 2024-09-17 08:23:20.142269829 +0000 UTC m=+0.488741090 container cleanup a7d7148fb0b5bb585900ca783c70061e264e98ce4b205b3a6745b1b0eb9c5aba (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T08:23:20.506 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:23:20 smithi047 bash[59629]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a 2024-09-17T08:23:20.506 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:23:20 smithi047 podman[59646]: 2024-09-17 08:23:20.301496776 +0000 UTC m=+0.566896688 container remove a7d7148fb0b5bb585900ca783c70061e264e98ce4b205b3a6745b1b0eb9c5aba (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T08:23:20.506 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:23:20 smithi047 systemd[1]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-17T08:23:20.789 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:23:20 smithi047 systemd[1]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-17T08:23:20.789 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:23:20 smithi047 systemd[1]: Stopped Ceph node-exporter.a for 3a66518a-74cd-11ef-bceb-c7b262605968. 2024-09-17T08:23:20.789 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 17 08:23:20 smithi047 systemd[1]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968@node-exporter.a.service: Consumed 1.868s CPU time. 2024-09-17T08:23:21.190 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-17T08:23:21.342 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:23:22.371 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:23:22 smithi193 systemd[1]: Stopping Ceph grafana.a for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:23:22.644 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:23:22 smithi193 podman[44456]: 2024-09-17 08:23:22.371534977 +0000 UTC m=+0.105354760 container died fe8eb604572a9f96ee8237a7444169db6f80fe347d920b61f74f0e44f0f5d9c6 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a, architecture=x86_64, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8, io.buildah.version=1.32.0, vcs-type=git, com.redhat.component=ubi8-container, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vendor=Red Hat, Inc., summary=Grafana Container configured for Ceph mgr/dashboard integration, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., distribution-scope=public, version=8.5, name=ubi8, description=Ceph Grafana Container, io.openshift.expose-services=, io.openshift.tags=base rhel8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, build-date=2022-04-29T11:48:11.055897, maintainer=Paul Cuzner ) 2024-09-17T08:23:22.984 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:23:22 smithi193 podman[44456]: 2024-09-17 08:23:22.725415688 +0000 UTC m=+0.459235515 container cleanup fe8eb604572a9f96ee8237a7444169db6f80fe347d920b61f74f0e44f0f5d9c6 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a, version=8.5, architecture=x86_64, build-date=2022-04-29T11:48:11.055897, description=Ceph Grafana Container, com.redhat.component=ubi8-container, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vendor=Red Hat, Inc., maintainer=Paul Cuzner , name=ubi8, distribution-scope=public, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, release=239.1651231664, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-type=git, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.buildah.version=1.32.0, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=) 2024-09-17T08:23:22.984 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:23:22 smithi193 bash[44456]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a 2024-09-17T08:23:23.336 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:23:22 smithi193 podman[44473]: 2024-09-17 08:23:22.984743089 +0000 UTC m=+0.615382877 container remove fe8eb604572a9f96ee8237a7444169db6f80fe347d920b61f74f0e44f0f5d9c6 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-grafana-a, build-date=2022-04-29T11:48:11.055897, io.openshift.tags=base rhel8, io.openshift.expose-services=, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, distribution-scope=public, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vcs-type=git, vendor=Red Hat, Inc., release=239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.buildah.version=1.32.0, version=8.5, description=Ceph Grafana Container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, architecture=x86_64, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., maintainer=Paul Cuzner , name=ubi8, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.component=ubi8-container) 2024-09-17T08:23:23.336 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:23:22 smithi193 systemd[1]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-17T08:23:23.337 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:23:23 smithi193 systemd[1]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-17T08:23:23.337 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:23:23 smithi193 systemd[1]: Stopped Ceph grafana.a for 3a66518a-74cd-11ef-bceb-c7b262605968. 2024-09-17T08:23:23.337 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 17 08:23:23 smithi193 systemd[1]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968@grafana.a.service: Consumed 2.961s CPU time. 2024-09-17T08:23:23.600 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:23:23 smithi193 systemd[1]: Stopping Ceph node-exporter.b for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:23:23.865 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:23:23 smithi193 podman[44588]: 2024-09-17 08:23:23.60018751 +0000 UTC m=+0.128027442 container died e6d4ab6a3ea3b97fc2bdf3f7e3e52bcc28e9c39ded30563e0f743f4c8f9da732 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T08:23:24.143 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:23:23 smithi193 podman[44588]: 2024-09-17 08:23:23.96442028 +0000 UTC m=+0.492260248 container cleanup e6d4ab6a3ea3b97fc2bdf3f7e3e52bcc28e9c39ded30563e0f743f4c8f9da732 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T08:23:24.144 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:23:23 smithi193 bash[44588]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b 2024-09-17T08:23:24.477 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:23:24 smithi193 podman[44607]: 2024-09-17 08:23:24.18189908 +0000 UTC m=+0.585679675 container remove e6d4ab6a3ea3b97fc2bdf3f7e3e52bcc28e9c39ded30563e0f743f4c8f9da732 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T08:23:24.477 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:23:24 smithi193 systemd[1]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-17T08:23:24.477 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:23:24 smithi193 systemd[1]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-17T08:23:24.477 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:23:24 smithi193 systemd[1]: Stopped Ceph node-exporter.b for 3a66518a-74cd-11ef-bceb-c7b262605968. 2024-09-17T08:23:24.477 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 17 08:23:24 smithi193 systemd[1]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968@node-exporter.b.service: Consumed 1.693s CPU time. 2024-09-17T08:23:24.979 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-17T08:23:25.131 INFO:teuthology.orchestra.run.smithi203.stdout:Deleting cluster with fsid: 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:23:26.034 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:23:25 smithi203 systemd[1]: Stopping Ceph node-exporter.c for 3a66518a-74cd-11ef-bceb-c7b262605968... 2024-09-17T08:23:26.034 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:23:25 smithi203 podman[44981]: 2024-09-17 08:23:25.98461604 +0000 UTC m=+0.100568884 container died a37d99d9c2e4209eb042c205d16cf3f47798de25580fde8890f6b1f817d1f4e9 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c, maintainer=The Prometheus Authors ) 2024-09-17T08:23:26.534 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:23:26 smithi203 podman[44981]: 2024-09-17 08:23:26.288403663 +0000 UTC m=+0.404356505 container cleanup a37d99d9c2e4209eb042c205d16cf3f47798de25580fde8890f6b1f817d1f4e9 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c, maintainer=The Prometheus Authors ) 2024-09-17T08:23:26.534 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:23:26 smithi203 bash[44981]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c 2024-09-17T08:23:26.534 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:23:26 smithi203 podman[44998]: 2024-09-17 08:23:26.455885491 +0000 UTC m=+0.475577645 container remove a37d99d9c2e4209eb042c205d16cf3f47798de25580fde8890f6b1f817d1f4e9 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3a66518a-74cd-11ef-bceb-c7b262605968-node-exporter-c, maintainer=The Prometheus Authors ) 2024-09-17T08:23:26.534 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:23:26 smithi203 systemd[1]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968@node-exporter.c.service: Main process exited, code=exited, status=143/n/a 2024-09-17T08:23:26.969 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:23:26 smithi203 systemd[1]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968@node-exporter.c.service: Failed with result 'exit-code'. 2024-09-17T08:23:26.969 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:23:26 smithi203 systemd[1]: Stopped Ceph node-exporter.c for 3a66518a-74cd-11ef-bceb-c7b262605968. 2024-09-17T08:23:26.969 INFO:journalctl@ceph.node-exporter.c.smithi203.stdout:Sep 17 08:23:26 smithi203 systemd[1]: ceph-3a66518a-74cd-11ef-bceb-c7b262605968@node-exporter.c.service: Consumed 1.706s CPU time. 2024-09-17T08:23:27.228 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:23:27.262 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:23:27.295 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:23:27.322 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-17T08:23:27.323 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908203/remote/smithi047/crash 2024-09-17T08:23:27.324 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/crash -- . 2024-09-17T08:23:27.369 INFO:teuthology.orchestra.run.smithi047.stderr:tar: /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-17T08:23:27.370 INFO:teuthology.orchestra.run.smithi047.stderr:tar: Error is not recoverable: exiting now 2024-09-17T08:23:27.372 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908203/remote/smithi193/crash 2024-09-17T08:23:27.373 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/crash -- . 2024-09-17T08:23:27.417 INFO:teuthology.orchestra.run.smithi193.stderr:tar: /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-17T08:23:27.418 INFO:teuthology.orchestra.run.smithi193.stderr:tar: Error is not recoverable: exiting now 2024-09-17T08:23:27.419 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908203/remote/smithi203/crash 2024-09-17T08:23:27.420 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/crash -- . 2024-09-17T08:23:27.454 INFO:teuthology.orchestra.run.smithi203.stderr:tar: /var/lib/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-17T08:23:27.454 INFO:teuthology.orchestra.run.smithi203.stderr:tar: Error is not recoverable: exiting now 2024-09-17T08:23:27.455 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-17T08:23:27.456 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v CEPHADM_AGENT_DOWN | egrep -v CEPHADM_FAILED_DAEMON | head -n 1 2024-09-17T08:23:27.487 INFO:teuthology.orchestra.run.smithi047.stderr:grep: /var/log/ceph/3a66518a-74cd-11ef-bceb-c7b262605968/ceph.log: No such file or directory 2024-09-17T08:23:27.489 INFO:tasks.cephadm:Compressing logs... 2024-09-17T08:23:27.490 DEBUG:teuthology.orchestra.run.smithi047:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T08:23:27.532 DEBUG:teuthology.orchestra.run.smithi193:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T08:23:27.535 DEBUG:teuthology.orchestra.run.smithi203:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T08:23:27.558 INFO:teuthology.orchestra.run.smithi203.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-17T08:23:27.559 INFO:teuthology.orchestra.run.smithi047.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-17T08:23:27.561 INFO:teuthology.orchestra.run.smithi193.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-17T08:23:28.166 INFO:tasks.cephadm:Archiving logs... 2024-09-17T08:23:28.166 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908203/remote/smithi047/log 2024-09-17T08:23:28.167 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T08:23:28.310 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908203/remote/smithi193/log 2024-09-17T08:23:28.311 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T08:23:28.392 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908203/remote/smithi203/log 2024-09-17T08:23:28.393 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T08:23:28.483 INFO:tasks.cephadm:Removing cluster... 2024-09-17T08:23:28.483 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 --force 2024-09-17T08:23:28.628 INFO:teuthology.orchestra.run.smithi047.stdout:Deleting cluster with fsid: 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:23:28.760 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 --force 2024-09-17T08:23:28.901 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:23:29.028 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3a66518a-74cd-11ef-bceb-c7b262605968 --force 2024-09-17T08:23:29.170 INFO:teuthology.orchestra.run.smithi203.stdout:Deleting cluster with fsid: 3a66518a-74cd-11ef-bceb-c7b262605968 2024-09-17T08:23:29.305 INFO:tasks.cephadm:Removing cephadm ... 2024-09-17T08:23:29.305 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-17T08:23:29.321 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-17T08:23:29.338 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-17T08:23:29.354 INFO:tasks.cephadm:Teardown complete 2024-09-17T08:23:29.355 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop 2024-09-17T08:23:29.362 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi047:/dev/vg_nvme/lv_1... 2024-09-17T08:23:29.363 DEBUG:teuthology.orchestra.run.smithi047:> sudo nvme disconnect -n lv_1 2024-09-17T08:23:29.509 INFO:teuthology.orchestra.run.smithi047.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-17T08:23:29.511 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi047:/dev/vg_nvme/lv_2... 2024-09-17T08:23:29.511 DEBUG:teuthology.orchestra.run.smithi047:> sudo nvme disconnect -n lv_2 2024-09-17T08:23:29.653 INFO:teuthology.orchestra.run.smithi047.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-17T08:23:29.655 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi047:/dev/vg_nvme/lv_3... 2024-09-17T08:23:29.655 DEBUG:teuthology.orchestra.run.smithi047:> sudo nvme disconnect -n lv_3 2024-09-17T08:23:29.817 INFO:teuthology.orchestra.run.smithi047.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-17T08:23:29.819 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi047:/dev/vg_nvme/lv_4... 2024-09-17T08:23:29.819 DEBUG:teuthology.orchestra.run.smithi047:> sudo nvme disconnect -n lv_4 2024-09-17T08:23:29.975 INFO:teuthology.orchestra.run.smithi047.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-17T08:23:29.977 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-17T08:23:29.977 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/scratch_devs 2024-09-17T08:23:30.002 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi193:/dev/vg_nvme/lv_1... 2024-09-17T08:23:30.003 DEBUG:teuthology.orchestra.run.smithi193:> sudo nvme disconnect -n lv_1 2024-09-17T08:23:30.206 INFO:teuthology.orchestra.run.smithi193.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-17T08:23:30.208 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi193:/dev/vg_nvme/lv_2... 2024-09-17T08:23:30.208 DEBUG:teuthology.orchestra.run.smithi193:> sudo nvme disconnect -n lv_2 2024-09-17T08:23:30.373 INFO:teuthology.orchestra.run.smithi193.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-17T08:23:30.375 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi193:/dev/vg_nvme/lv_3... 2024-09-17T08:23:30.375 DEBUG:teuthology.orchestra.run.smithi193:> sudo nvme disconnect -n lv_3 2024-09-17T08:23:30.560 INFO:teuthology.orchestra.run.smithi193.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-17T08:23:30.562 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi193:/dev/vg_nvme/lv_4... 2024-09-17T08:23:30.562 DEBUG:teuthology.orchestra.run.smithi193:> sudo nvme disconnect -n lv_4 2024-09-17T08:23:30.760 INFO:teuthology.orchestra.run.smithi193.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-17T08:23:30.762 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-17T08:23:30.762 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/scratch_devs 2024-09-17T08:23:30.786 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi203:/dev/vg_nvme/lv_1... 2024-09-17T08:23:30.786 DEBUG:teuthology.orchestra.run.smithi203:> sudo nvme disconnect -n lv_1 2024-09-17T08:23:30.934 INFO:teuthology.orchestra.run.smithi203.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-17T08:23:30.936 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi203:/dev/vg_nvme/lv_2... 2024-09-17T08:23:30.936 DEBUG:teuthology.orchestra.run.smithi203:> sudo nvme disconnect -n lv_2 2024-09-17T08:23:31.076 INFO:teuthology.orchestra.run.smithi203.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-17T08:23:31.077 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi203:/dev/vg_nvme/lv_3... 2024-09-17T08:23:31.078 DEBUG:teuthology.orchestra.run.smithi203:> sudo nvme disconnect -n lv_3 2024-09-17T08:23:31.230 INFO:teuthology.orchestra.run.smithi203.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-17T08:23:31.232 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi203:/dev/vg_nvme/lv_4... 2024-09-17T08:23:31.232 DEBUG:teuthology.orchestra.run.smithi203:> sudo nvme disconnect -n lv_4 2024-09-17T08:23:31.383 INFO:teuthology.orchestra.run.smithi203.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-17T08:23:31.384 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-17T08:23:31.384 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/scratch_devs 2024-09-17T08:23:31.409 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T08:23:31.419 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T08:23:31.419 DEBUG:teuthology.orchestra.run.smithi047:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T08:23:31.422 DEBUG:teuthology.orchestra.run.smithi193:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T08:23:31.425 DEBUG:teuthology.orchestra.run.smithi203:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T08:23:31.436 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-09-17T08:23:31.437 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-09-17T08:23:31.441 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T08:23:31.441 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-09-17T08:23:31.441 INFO:teuthology.orchestra.run.smithi193.stdout:^* hv01.front.sepia.ceph.com 3 6 377 6 +638us[ +647us] +/- 47ms 2024-09-17T08:23:31.441 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 3 -3131us[-3131us] +/- 54ms 2024-09-17T08:23:31.442 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 +2980us[+2980us] +/- 67ms 2024-09-17T08:23:31.442 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:23:31.442 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T08:23:31.443 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-09-17T08:23:31.443 INFO:teuthology.orchestra.run.smithi047.stdout:^* hv01.front.sepia.ceph.com 3 6 377 4 +655us[ +668us] +/- 47ms 2024-09-17T08:23:31.443 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 -3101us[-3089us] +/- 53ms 2024-09-17T08:23:31.443 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +3012us[+3025us] +/- 67ms 2024-09-17T08:23:31.443 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:23:31.464 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-09-17T08:23:31.468 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T08:23:31.468 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-09-17T08:23:31.468 INFO:teuthology.orchestra.run.smithi203.stdout:^* hv01.front.sepia.ceph.com 3 6 377 5 +651us[ +673us] +/- 47ms 2024-09-17T08:23:31.468 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 4 -3091us[-3091us] +/- 53ms 2024-09-17T08:23:31.468 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +3013us[+3013us] +/- 67ms 2024-09-17T08:23:31.468 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:23:31.469 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T08:23:31.566 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T08:23:31.567 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T08:23:31.631 DEBUG:teuthology.orchestra.run.smithi047:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T08:23:31.712 DEBUG:teuthology.orchestra.run.smithi193:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T08:23:31.776 DEBUG:teuthology.orchestra.run.smithi203:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T08:23:31.841 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T08:23:31.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:31.874 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T08:23:31.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:31.902 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T08:23:31.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:31.929 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T08:23:31.938 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T08:23:32.008 INFO:teuthology.task.internal:Duration was 627.171515 seconds 2024-09-17T08:23:32.008 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T08:23:32.108 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T08:23:32.108 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T08:23:32.111 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T08:23:32.114 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T08:23:32.150 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T08:23:32.152 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T08:23:32.155 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T08:23:32.581 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T08:23:32.581 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi047.front.sepia.ceph.com 2024-09-17T08:23:32.581 DEBUG:teuthology.orchestra.run.smithi047:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T08:23:32.605 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi193.front.sepia.ceph.com 2024-09-17T08:23:32.605 DEBUG:teuthology.orchestra.run.smithi193:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T08:23:32.634 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi203.front.sepia.ceph.com 2024-09-17T08:23:32.634 DEBUG:teuthology.orchestra.run.smithi203:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T08:23:32.657 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T08:23:32.657 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T08:23:32.660 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T08:23:32.677 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T08:23:32.724 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T08:23:32.724 DEBUG:teuthology.orchestra.run.smithi047:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T08:23:32.729 DEBUG:teuthology.orchestra.run.smithi193:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T08:23:32.743 DEBUG:teuthology.orchestra.run.smithi203:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T08:23:32.931 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T08:23:32.941 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T08:23:32.941 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T08:23:32.967 DEBUG:teuthology.orchestra.run.smithi193:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T08:23:32.992 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T08:23:33.017 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T08:23:33.117 DEBUG:teuthology.orchestra.run.smithi047:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T08:23:33.120 DEBUG:teuthology.orchestra.run.smithi193:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T08:23:33.122 DEBUG:teuthology.orchestra.run.smithi203:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T08:23:33.142 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = core 2024-09-17T08:23:33.142 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = core 2024-09-17T08:23:33.145 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = core 2024-09-17T08:23:33.179 DEBUG:teuthology.orchestra.run.smithi047:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T08:23:33.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:33.235 DEBUG:teuthology.orchestra.run.smithi193:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T08:23:33.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:33.250 DEBUG:teuthology.orchestra.run.smithi203:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T08:23:33.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:33.264 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T08:23:33.274 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T08:23:33.275 DEBUG:teuthology.misc:Transferring archived files from smithi047:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908203/remote/smithi047 2024-09-17T08:23:33.275 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T08:23:33.312 DEBUG:teuthology.misc:Transferring archived files from smithi193:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908203/remote/smithi193 2024-09-17T08:23:33.313 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T08:23:33.348 DEBUG:teuthology.misc:Transferring archived files from smithi203:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908203/remote/smithi203 2024-09-17T08:23:33.348 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T08:23:33.389 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T08:23:33.389 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T08:23:33.391 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T08:23:33.394 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T08:23:33.437 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T08:23:33.447 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T08:23:33.448 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T08:23:33.508 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T08:23:33.508 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T08:23:33.510 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T08:23:33.512 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T08:23:33.523 INFO:teuthology.orchestra.run.smithi047.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 08:23 /home/ubuntu/cephtest 2024-09-17T08:23:33.525 INFO:teuthology.orchestra.run.smithi193.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 08:23 /home/ubuntu/cephtest 2024-09-17T08:23:33.527 INFO:teuthology.orchestra.run.smithi203.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 08:23 /home/ubuntu/cephtest 2024-09-17T08:23:33.528 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T08:23:33.610 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T08:23:33.803 INFO:teuthology.run:Summary data: description: orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} duration: 627.1715149879456 owner: scheduled_teuthology@teuthology success: true 2024-09-17T08:23:33.803 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T08:23:33.934 INFO:teuthology.run:pass