2024-09-17T09:11:28.350 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T09:11:28.365 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T09:11:28.448 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908293 branch: reef description: orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} email: ceph-qa@ceph.com first_in_suite: false job_id: '7908293' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 1 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: reef ceph: conf: global: mon election default strategy: 3 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: false mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - CEPHADM_AGENT_DOWN - CEPHADM_FAILED_DAEMON log-only-match: - CEPHADM_ sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: reef sha1: 1cbdc9211df088cccbd94423c7cf3a2b85fe432b owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - osd.0 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.1 - client.1 - prometheus.a - grafana.a - node-exporter.b - - mon.c - mgr.z - osd.2 - client.2 - node-exporter.c seed: 5234 sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 sleep_before_teardown: 0 subset: 46/64 suite: orch suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_1cbdc9211df088cccbd94423c7cf3a2b85fe432b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 1cbdc9211df088cccbd94423c7cf3a2b85fe432b targets: smithi003.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJFDaYgwz7I0OSWMuz/BIfnuCwbn+fTblvXgRPDYMe3/02h3/YtRaNkCg3Di19s49DUYjN2RW8V2bP67CmzDPwI= smithi077.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP6jxIkNAD1jNy4uAhoaDeeSeJe61mLYhPaPI6IVQMNHFo2cKh/7puk5NpfT48AW/xuX9swbYJ9ngWkRVQXxc+I= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDUVvu4Ab+ItgG7cS/skYAIvnGp0kV1BAL7nCQ/BXmAkGRGfvmafJDnWJ+Tvw4H0lSHcAX+tI6UF2mDxvuu5Lb4= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install runc nvmetcli nvme-cli -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - nvme_loop: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - stat -c '%u %g' /var/log/ceph | grep '167 167' - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - ceph orch ls --format yaml - ceph orch ls | grep '^osd ' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_22:08:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T09:11:28.448 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_1cbdc9211df088cccbd94423c7cf3a2b85fe432b/qa; will attempt to use it 2024-09-17T09:11:28.449 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_1cbdc9211df088cccbd94423c7cf3a2b85fe432b/qa/tasks 2024-09-17T09:11:28.449 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T09:11:28.451 INFO:teuthology.task.internal:Checking packages... 2024-09-17T09:11:28.473 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9' 2024-09-17T09:11:28.473 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T09:11:28.473 INFO:teuthology.packaging:ref: None 2024-09-17T09:11:28.473 INFO:teuthology.packaging:tag: None 2024-09-17T09:11:28.473 INFO:teuthology.packaging:branch: reef 2024-09-17T09:11:28.473 INFO:teuthology.packaging:sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T09:11:28.474 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-17T09:11:28.764 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-855.g19e2a2be 2024-09-17T09:11:28.765 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T09:11:28.772 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T09:11:28.772 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T09:11:28.836 INFO:teuthology.task.internal:Saving configuration 2024-09-17T09:11:28.849 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T09:11:28.904 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T09:11:28.928 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908293', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 09:06:28.022617', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJFDaYgwz7I0OSWMuz/BIfnuCwbn+fTblvXgRPDYMe3/02h3/YtRaNkCg3Di19s49DUYjN2RW8V2bP67CmzDPwI='} 2024-09-17T09:11:28.952 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi077.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908293', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 09:06:28.024044', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP6jxIkNAD1jNy4uAhoaDeeSeJe61mLYhPaPI6IVQMNHFo2cKh/7puk5NpfT48AW/xuX9swbYJ9ngWkRVQXxc+I='} 2024-09-17T09:11:28.974 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908293', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 09:06:28.021134', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDUVvu4Ab+ItgG7cS/skYAIvnGp0kV1BAL7nCQ/BXmAkGRGfvmafJDnWJ+Tvw4H0lSHcAX+tI6UF2mDxvuu5Lb4='} 2024-09-17T09:11:28.974 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T09:11:28.981 INFO:teuthology.task.internal:roles: ubuntu@smithi003.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-17T09:11:28.981 INFO:teuthology.task.internal:roles: ubuntu@smithi077.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.1', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2024-09-17T09:11:28.981 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'osd.2', 'client.2', 'node-exporter.c'] 2024-09-17T09:11:28.981 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T09:11:29.117 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2bebd8bac0>, signals=[15]) 2024-09-17T09:11:29.117 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T09:11:29.123 INFO:teuthology.task.internal:Opening connections... 2024-09-17T09:11:29.123 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi003.front.sepia.ceph.com 2024-09-17T09:11:29.125 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:11:29.213 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi077.front.sepia.ceph.com 2024-09-17T09:11:29.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:11:29.303 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2024-09-17T09:11:29.303 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:11:29.405 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T09:11:29.412 DEBUG:teuthology.orchestra.run.smithi003:> uname -m 2024-09-17T09:11:29.430 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64 2024-09-17T09:11:29.430 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2024-09-17T09:11:29.488 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="CentOS Stream" 2024-09-17T09:11:29.488 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="9" 2024-09-17T09:11:29.489 INFO:teuthology.orchestra.run.smithi003.stdout:ID="centos" 2024-09-17T09:11:29.489 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="rhel fedora" 2024-09-17T09:11:29.489 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="9" 2024-09-17T09:11:29.489 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el9" 2024-09-17T09:11:29.489 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T09:11:29.489 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31" 2024-09-17T09:11:29.489 INFO:teuthology.orchestra.run.smithi003.stdout:LOGO="fedora-logo-icon" 2024-09-17T09:11:29.489 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T09:11:29.489 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://centos.org/" 2024-09-17T09:11:29.489 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T09:11:29.489 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T09:11:29.489 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T09:11:29.490 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2024-09-17T09:11:29.510 DEBUG:teuthology.orchestra.run.smithi077:> uname -m 2024-09-17T09:11:29.528 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64 2024-09-17T09:11:29.528 DEBUG:teuthology.orchestra.run.smithi077:> cat /etc/os-release 2024-09-17T09:11:29.586 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="CentOS Stream" 2024-09-17T09:11:29.586 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="9" 2024-09-17T09:11:29.586 INFO:teuthology.orchestra.run.smithi077.stdout:ID="centos" 2024-09-17T09:11:29.586 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="rhel fedora" 2024-09-17T09:11:29.586 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="9" 2024-09-17T09:11:29.586 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el9" 2024-09-17T09:11:29.586 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T09:11:29.586 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31" 2024-09-17T09:11:29.587 INFO:teuthology.orchestra.run.smithi077.stdout:LOGO="fedora-logo-icon" 2024-09-17T09:11:29.587 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T09:11:29.587 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://centos.org/" 2024-09-17T09:11:29.587 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T09:11:29.587 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T09:11:29.587 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T09:11:29.587 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2024-09-17T09:11:29.610 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2024-09-17T09:11:29.628 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2024-09-17T09:11:29.629 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-09-17T09:11:29.686 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2024-09-17T09:11:29.687 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2024-09-17T09:11:29.687 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2024-09-17T09:11:29.687 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2024-09-17T09:11:29.687 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2024-09-17T09:11:29.687 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2024-09-17T09:11:29.687 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T09:11:29.687 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2024-09-17T09:11:29.687 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2024-09-17T09:11:29.687 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T09:11:29.687 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2024-09-17T09:11:29.687 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T09:11:29.687 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T09:11:29.688 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T09:11:29.688 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-09-17T09:11:29.762 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T09:11:29.771 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T09:11:29.816 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T09:11:29.817 DEBUG:teuthology.orchestra.run.smithi003:> test '!' -e /home/ubuntu/cephtest 2024-09-17T09:11:29.819 DEBUG:teuthology.orchestra.run.smithi077:> test '!' -e /home/ubuntu/cephtest 2024-09-17T09:11:29.822 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2024-09-17T09:11:29.839 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T09:11:29.866 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T09:11:29.867 DEBUG:teuthology.orchestra.run.smithi003:> test -z $(ls -A /var/lib/ceph) 2024-09-17T09:11:29.876 DEBUG:teuthology.orchestra.run.smithi077:> test -z $(ls -A /var/lib/ceph) 2024-09-17T09:11:29.880 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2024-09-17T09:11:29.954 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T09:11:30.035 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T09:11:30.049 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-17T09:11:30.050 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-17T09:11:30.050 DEBUG:teuthology.orchestra.run.smithi003:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T09:11:30.052 DEBUG:teuthology.orchestra.run.smithi077:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T09:11:30.052 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T09:11:30.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T09:11:30.069 DEBUG:teuthology.orchestra.run.smithi077:> uname -r 2024-09-17T09:11:30.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T09:11:30.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T09:11:30.071 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2024-09-17T09:11:30.071 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-09-17T09:11:30.127 INFO:teuthology.orchestra.run.smithi077.stdout:5.14.0-503.el9.x86_64 2024-09-17T09:11:30.127 INFO:teuthology.task.kernel:Running kernel on smithi077: 5.14.0-503.el9.x86_64 2024-09-17T09:11:30.128 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum install -y kernel 2024-09-17T09:11:30.129 INFO:teuthology.orchestra.run.smithi183.stdout:5.14.0-503.el9.x86_64 2024-09-17T09:11:30.130 INFO:teuthology.task.kernel:Running kernel on smithi183: 5.14.0-503.el9.x86_64 2024-09-17T09:11:30.130 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum install -y kernel 2024-09-17T09:11:30.131 INFO:teuthology.orchestra.run.smithi003.stdout:5.14.0-503.el9.x86_64 2024-09-17T09:11:30.131 INFO:teuthology.task.kernel:Running kernel on smithi003: 5.14.0-503.el9.x86_64 2024-09-17T09:11:30.131 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum install -y kernel 2024-09-17T09:11:32.673 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.3 MB 00:00 2024-09-17T09:11:33.603 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.3 MB 00:01 2024-09-17T09:11:33.653 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.3 MB 00:01 2024-09-17T09:11:38.846 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - AppStream 4.9 MB/s | 20 MB 00:04 2024-09-17T09:11:39.104 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - AppStream 4.6 MB/s | 20 MB 00:04 2024-09-17T09:11:44.404 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.5 MB 00:01 2024-09-17T09:11:45.555 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.5 MB 00:02 2024-09-17T09:11:46.446 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 19 kB 00:00 2024-09-17T09:11:47.189 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 19 kB 00:00 2024-09-17T09:11:47.259 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-17T09:11:48.002 INFO:teuthology.orchestra.run.smithi077.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-17T09:11:52.890 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-09-17T09:11:53.416 INFO:teuthology.orchestra.run.smithi077.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-17T09:11:54.043 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 20 MB 00:20 2024-09-17T09:11:54.855 INFO:teuthology.orchestra.run.smithi003.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T09:11:54.855 INFO:teuthology.orchestra.run.smithi003.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T09:11:54.950 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-17T09:11:54.957 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-17T09:11:54.957 INFO:teuthology.orchestra.run.smithi003.stdout: Package Architecture Version Repository Size 2024-09-17T09:11:54.957 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-17T09:11:54.957 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-17T09:11:54.957 INFO:teuthology.orchestra.run.smithi003.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T09:11:54.957 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-09-17T09:11:54.957 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T09:11:54.957 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T09:11:54.957 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T09:11:54.957 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:11:54.958 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-17T09:11:54.958 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-17T09:11:54.958 INFO:teuthology.orchestra.run.smithi003.stdout:Install 4 Packages 2024-09-17T09:11:54.958 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:11:54.958 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 79 M 2024-09-17T09:11:54.958 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 126 M 2024-09-17T09:11:54.958 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-17T09:11:55.322 INFO:teuthology.orchestra.run.smithi077.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T09:11:55.322 INFO:teuthology.orchestra.run.smithi077.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T09:11:55.366 INFO:teuthology.orchestra.run.smithi003.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 99 kB/s | 36 kB 00:00 2024-09-17T09:11:55.426 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-09-17T09:11:55.436 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-09-17T09:11:55.436 INFO:teuthology.orchestra.run.smithi077.stdout: Package Architecture Version Repository Size 2024-09-17T09:11:55.436 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-09-17T09:11:55.436 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-09-17T09:11:55.436 INFO:teuthology.orchestra.run.smithi077.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T09:11:55.436 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-09-17T09:11:55.437 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T09:11:55.437 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T09:11:55.437 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T09:11:55.437 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-17T09:11:55.437 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-09-17T09:11:55.437 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-09-17T09:11:55.437 INFO:teuthology.orchestra.run.smithi077.stdout:Install 4 Packages 2024-09-17T09:11:55.437 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-17T09:11:55.438 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 79 M 2024-09-17T09:11:55.438 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 126 M 2024-09-17T09:11:55.438 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-09-17T09:11:55.677 INFO:teuthology.orchestra.run.smithi077.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 368 kB/s | 36 kB 00:00 2024-09-17T09:11:56.068 INFO:teuthology.orchestra.run.smithi077.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-17T09:11:56.526 INFO:teuthology.orchestra.run.smithi077.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 36 MB/s | 35 MB 00:00 2024-09-17T09:11:56.835 INFO:teuthology.orchestra.run.smithi077.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 25 MB/s | 29 MB 00:01 2024-09-17T09:11:56.835 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-09-17T09:11:56.835 INFO:teuthology.orchestra.run.smithi077.stdout:Total 56 MB/s | 79 MB 00:01 2024-09-17T09:11:57.521 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-09-17T09:11:57.874 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-09-17T09:11:57.874 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-09-17T09:11:58.366 INFO:teuthology.orchestra.run.smithi003.stdout:(2/4): kernel-modules-core-5.14.0-508.el9.x86_6 9.5 MB/s | 29 MB 00:02 2024-09-17T09:11:58.418 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-17T09:11:58.831 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-09-17T09:11:58.831 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-09-17T09:11:59.861 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 81 kB/s | 19 kB 00:00 2024-09-17T09:12:01.006 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-17T09:12:01.273 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-09-17T09:12:01.923 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T09:12:02.003 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T09:12:04.780 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T09:12:04.916 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T09:12:06.371 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-17T09:12:08.294 INFO:teuthology.orchestra.run.smithi183.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T09:12:08.294 INFO:teuthology.orchestra.run.smithi183.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T09:12:08.377 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-09-17T09:12:08.390 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-09-17T09:12:08.390 INFO:teuthology.orchestra.run.smithi183.stdout: Package Architecture Version Repository Size 2024-09-17T09:12:08.390 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-09-17T09:12:08.390 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-09-17T09:12:08.390 INFO:teuthology.orchestra.run.smithi183.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T09:12:08.390 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-09-17T09:12:08.390 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T09:12:08.390 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T09:12:08.390 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T09:12:08.390 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-17T09:12:08.391 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-09-17T09:12:08.391 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-09-17T09:12:08.391 INFO:teuthology.orchestra.run.smithi183.stdout:Install 4 Packages 2024-09-17T09:12:08.391 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-17T09:12:08.391 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 79 M 2024-09-17T09:12:08.391 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 126 M 2024-09-17T09:12:08.392 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-09-17T09:12:08.613 INFO:teuthology.orchestra.run.smithi183.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 395 kB/s | 36 kB 00:00 2024-09-17T09:12:08.972 INFO:teuthology.orchestra.run.smithi183.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-09-17T09:12:09.289 INFO:teuthology.orchestra.run.smithi183.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 42 MB/s | 29 MB 00:00 2024-09-17T09:12:09.728 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T09:12:09.773 INFO:teuthology.orchestra.run.smithi183.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 28 MB/s | 35 MB 00:01 2024-09-17T09:12:09.773 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-09-17T09:12:09.773 INFO:teuthology.orchestra.run.smithi183.stdout:Total 57 MB/s | 79 MB 00:01 2024-09-17T09:12:09.775 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:12:10.452 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-09-17T09:12:10.793 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-09-17T09:12:10.793 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-09-17T09:12:11.773 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-09-17T09:12:11.773 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-09-17T09:12:14.207 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-09-17T09:12:14.634 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:12:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T09:12:14.941 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T09:12:17.331 INFO:teuthology.orchestra.run.smithi003.stdout:(3/4): kernel-core-5.14.0-508.el9.x86_64.rpm 718 kB/s | 16 MB 00:22 2024-09-17T09:12:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T09:12:17.816 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T09:12:22.653 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T09:12:22.766 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:12:27.592 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:12:47.594 INFO:teuthology.orchestra.run.smithi003.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 672 kB/s | 35 MB 00:52 2024-09-17T09:12:47.595 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-17T09:12:47.595 INFO:teuthology.orchestra.run.smithi003.stdout:Total 1.5 MB/s | 79 MB 00:52 2024-09-17T09:12:48.304 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-17T09:12:48.676 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-17T09:12:48.676 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-17T09:12:49.645 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-17T09:12:49.645 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-17T09:12:52.226 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-17T09:12:52.915 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T09:12:52.996 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T09:12:53.012 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:12:53.061 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:12:53.452 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:12:53.453 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T09:12:53.453 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T09:12:53.453 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T09:12:54.289 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:12:54.289 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-17T09:12:54.289 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-09-17T09:12:54.289 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T09:12:54.289 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T09:12:54.289 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T09:12:54.289 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T09:12:54.289 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-17T09:12:54.289 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-09-17T09:12:54.662 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T09:12:54.735 INFO:teuthology.orchestra.run.smithi077.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T09:12:54.736 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T09:12:54.736 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T09:12:54.737 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T09:12:54.737 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T09:12:54.737 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-17T09:12:54.737 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi077.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T09:12:54.737 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T09:12:54.796 INFO:teuthology.orchestra.run.smithi077.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T09:12:54.796 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T09:12:54.797 DEBUG:teuthology.orchestra.run.smithi077:> sudo rpm -qi grub2-tools 2024-09-17T09:12:54.828 INFO:teuthology.orchestra.run.smithi077.stdout:Name : grub2-tools 2024-09-17T09:12:54.829 INFO:teuthology.orchestra.run.smithi077.stdout:Epoch : 1 2024-09-17T09:12:54.829 INFO:teuthology.orchestra.run.smithi077.stdout:Version : 2.06 2024-09-17T09:12:54.829 INFO:teuthology.orchestra.run.smithi077.stdout:Release : 82.el9 2024-09-17T09:12:54.829 INFO:teuthology.orchestra.run.smithi077.stdout:Architecture: x86_64 2024-09-17T09:12:54.829 INFO:teuthology.orchestra.run.smithi077.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T09:12:54.829 INFO:teuthology.orchestra.run.smithi077.stdout:Group : Unspecified 2024-09-17T09:12:54.829 INFO:teuthology.orchestra.run.smithi077.stdout:Size : 8274670 2024-09-17T09:12:54.829 INFO:teuthology.orchestra.run.smithi077.stdout:License : GPLv3+ 2024-09-17T09:12:54.829 INFO:teuthology.orchestra.run.smithi077.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T09:12:54.829 INFO:teuthology.orchestra.run.smithi077.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T09:12:54.829 INFO:teuthology.orchestra.run.smithi077.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T09:12:54.830 INFO:teuthology.orchestra.run.smithi077.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T09:12:54.830 INFO:teuthology.orchestra.run.smithi077.stdout:Packager : builder@centos.org 2024-09-17T09:12:54.830 INFO:teuthology.orchestra.run.smithi077.stdout:Vendor : CentOS 2024-09-17T09:12:54.830 INFO:teuthology.orchestra.run.smithi077.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T09:12:54.830 INFO:teuthology.orchestra.run.smithi077.stdout:Summary : Support tools for GRUB. 2024-09-17T09:12:54.830 INFO:teuthology.orchestra.run.smithi077.stdout:Description : 2024-09-17T09:12:54.830 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-17T09:12:54.830 INFO:teuthology.orchestra.run.smithi077.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T09:12:54.830 INFO:teuthology.orchestra.run.smithi077.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T09:12:54.830 INFO:teuthology.orchestra.run.smithi077.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T09:12:54.830 INFO:teuthology.orchestra.run.smithi077.stdout:hardware devices. 2024-09-17T09:12:54.830 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-17T09:12:54.830 INFO:teuthology.orchestra.run.smithi077.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T09:12:54.831 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T09:12:54.831 INFO:teuthology.task.kernel:Updating grub on smithi077 to boot 5.14.0-508.el9.x86_64 2024-09-17T09:12:54.832 DEBUG:teuthology.orchestra.run.smithi077:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T09:12:55.464 INFO:teuthology.orchestra.run.smithi077.stderr:Generating grub configuration file ... 2024-09-17T09:12:55.743 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T09:12:55.873 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T09:12:56.793 INFO:teuthology.orchestra.run.smithi077.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T09:12:56.819 INFO:teuthology.orchestra.run.smithi077.stderr:done 2024-09-17T09:12:56.821 DEBUG:teuthology.orchestra.run.smithi077:> mktemp 2024-09-17T09:12:56.837 INFO:teuthology.orchestra.run.smithi077.stdout:/tmp/tmp.UL9eM5a01L 2024-09-17T09:12:56.838 DEBUG:teuthology.orchestra.run.smithi077:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UL9eM5a01L 2024-09-17T09:12:56.905 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0666 /tmp/tmp.UL9eM5a01L 2024-09-17T09:12:57.052 DEBUG:teuthology.orchestra.remote:smithi077:/tmp/tmp.UL9eM5a01L is 6KB 2024-09-17T09:12:57.102 DEBUG:teuthology.orchestra.run.smithi077:> rm -fr /tmp/tmp.UL9eM5a01L 2024-09-17T09:12:57.117 DEBUG:teuthology.orchestra.run.smithi077:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T09:12:57.180 INFO:teuthology.orchestra.run.smithi077.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T09:12:57.180 INFO:teuthology.orchestra.run.smithi077.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T09:12:57.180 INFO:teuthology.orchestra.run.smithi077.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T09:12:57.180 INFO:teuthology.orchestra.run.smithi077.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T09:12:57.181 DEBUG:teuthology.orchestra.run.smithi077:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T09:12:57.350 DEBUG:teuthology.orchestra.run.smithi077:> sudo shutdown -r now 2024-09-17T09:13:00.994 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T09:13:01.040 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:13:06.172 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:13:06.174 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:13:06.220 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:13:06.616 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:13:06.616 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T09:13:06.616 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T09:13:06.616 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T09:13:07.470 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:13:07.470 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-17T09:13:07.470 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-09-17T09:13:07.470 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T09:13:07.470 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T09:13:07.470 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T09:13:07.470 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T09:13:07.470 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-17T09:13:07.471 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-09-17T09:13:07.814 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T09:13:07.896 INFO:teuthology.orchestra.run.smithi183.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T09:13:07.896 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T09:13:07.896 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T09:13:07.897 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T09:13:07.897 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T09:13:07.897 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-17T09:13:07.897 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi183.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T09:13:07.897 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T09:13:07.955 INFO:teuthology.orchestra.run.smithi183.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T09:13:07.955 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T09:13:07.956 DEBUG:teuthology.orchestra.run.smithi183:> sudo rpm -qi grub2-tools 2024-09-17T09:13:07.992 INFO:teuthology.orchestra.run.smithi183.stdout:Name : grub2-tools 2024-09-17T09:13:07.993 INFO:teuthology.orchestra.run.smithi183.stdout:Epoch : 1 2024-09-17T09:13:07.993 INFO:teuthology.orchestra.run.smithi183.stdout:Version : 2.06 2024-09-17T09:13:07.993 INFO:teuthology.orchestra.run.smithi183.stdout:Release : 82.el9 2024-09-17T09:13:07.993 INFO:teuthology.orchestra.run.smithi183.stdout:Architecture: x86_64 2024-09-17T09:13:07.993 INFO:teuthology.orchestra.run.smithi183.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T09:13:07.993 INFO:teuthology.orchestra.run.smithi183.stdout:Group : Unspecified 2024-09-17T09:13:07.993 INFO:teuthology.orchestra.run.smithi183.stdout:Size : 8274670 2024-09-17T09:13:07.993 INFO:teuthology.orchestra.run.smithi183.stdout:License : GPLv3+ 2024-09-17T09:13:07.993 INFO:teuthology.orchestra.run.smithi183.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T09:13:07.993 INFO:teuthology.orchestra.run.smithi183.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T09:13:07.993 INFO:teuthology.orchestra.run.smithi183.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T09:13:07.994 INFO:teuthology.orchestra.run.smithi183.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T09:13:07.994 INFO:teuthology.orchestra.run.smithi183.stdout:Packager : builder@centos.org 2024-09-17T09:13:07.994 INFO:teuthology.orchestra.run.smithi183.stdout:Vendor : CentOS 2024-09-17T09:13:07.994 INFO:teuthology.orchestra.run.smithi183.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T09:13:07.994 INFO:teuthology.orchestra.run.smithi183.stdout:Summary : Support tools for GRUB. 2024-09-17T09:13:07.994 INFO:teuthology.orchestra.run.smithi183.stdout:Description : 2024-09-17T09:13:07.994 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-17T09:13:07.994 INFO:teuthology.orchestra.run.smithi183.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T09:13:07.994 INFO:teuthology.orchestra.run.smithi183.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T09:13:07.994 INFO:teuthology.orchestra.run.smithi183.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T09:13:07.994 INFO:teuthology.orchestra.run.smithi183.stdout:hardware devices. 2024-09-17T09:13:07.994 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-17T09:13:07.994 INFO:teuthology.orchestra.run.smithi183.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T09:13:07.995 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T09:13:07.995 INFO:teuthology.task.kernel:Updating grub on smithi183 to boot 5.14.0-508.el9.x86_64 2024-09-17T09:13:07.996 DEBUG:teuthology.orchestra.run.smithi183:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T09:13:08.726 INFO:teuthology.orchestra.run.smithi183.stderr:Generating grub configuration file ... 2024-09-17T09:13:10.045 INFO:teuthology.orchestra.run.smithi183.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T09:13:10.081 INFO:teuthology.orchestra.run.smithi183.stderr:done 2024-09-17T09:13:10.083 DEBUG:teuthology.orchestra.run.smithi183:> mktemp 2024-09-17T09:13:10.098 INFO:teuthology.orchestra.run.smithi183.stdout:/tmp/tmp.ljcD2aSJKJ 2024-09-17T09:13:10.099 DEBUG:teuthology.orchestra.run.smithi183:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ljcD2aSJKJ 2024-09-17T09:13:10.165 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0666 /tmp/tmp.ljcD2aSJKJ 2024-09-17T09:13:10.311 DEBUG:teuthology.orchestra.remote:smithi183:/tmp/tmp.ljcD2aSJKJ is 6KB 2024-09-17T09:13:10.360 DEBUG:teuthology.orchestra.run.smithi183:> rm -fr /tmp/tmp.ljcD2aSJKJ 2024-09-17T09:13:10.375 DEBUG:teuthology.orchestra.run.smithi183:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T09:13:10.438 INFO:teuthology.orchestra.run.smithi183.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T09:13:10.438 INFO:teuthology.orchestra.run.smithi183.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T09:13:10.438 INFO:teuthology.orchestra.run.smithi183.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T09:13:10.438 INFO:teuthology.orchestra.run.smithi183.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T09:13:10.440 DEBUG:teuthology.orchestra.run.smithi183:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T09:13:10.605 DEBUG:teuthology.orchestra.run.smithi183:> sudo shutdown -r now 2024-09-17T09:13:27.353 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-17T09:13:27.353 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-09-17T09:13:27.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:13:40.608 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-17T09:13:40.609 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-17T09:13:40.609 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:13:45.519 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:13:45.558 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:13:45.943 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:13:45.943 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T09:13:45.943 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T09:13:45.943 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T09:13:46.789 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:13:46.789 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:13:46.790 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-17T09:13:46.790 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T09:13:46.790 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T09:13:46.790 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T09:13:46.790 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T09:13:46.790 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:13:46.790 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-17T09:13:47.137 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T09:13:47.231 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T09:13:47.231 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T09:13:47.231 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T09:13:47.231 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T09:13:47.231 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T09:13:47.232 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-17T09:13:47.232 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi003.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T09:13:47.232 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T09:13:47.290 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T09:13:47.291 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T09:13:47.291 DEBUG:teuthology.orchestra.run.smithi003:> sudo rpm -qi grub2-tools 2024-09-17T09:13:47.322 INFO:teuthology.orchestra.run.smithi003.stdout:Name : grub2-tools 2024-09-17T09:13:47.322 INFO:teuthology.orchestra.run.smithi003.stdout:Epoch : 1 2024-09-17T09:13:47.323 INFO:teuthology.orchestra.run.smithi003.stdout:Version : 2.06 2024-09-17T09:13:47.323 INFO:teuthology.orchestra.run.smithi003.stdout:Release : 82.el9 2024-09-17T09:13:47.323 INFO:teuthology.orchestra.run.smithi003.stdout:Architecture: x86_64 2024-09-17T09:13:47.323 INFO:teuthology.orchestra.run.smithi003.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T09:13:47.323 INFO:teuthology.orchestra.run.smithi003.stdout:Group : Unspecified 2024-09-17T09:13:47.323 INFO:teuthology.orchestra.run.smithi003.stdout:Size : 8274670 2024-09-17T09:13:47.323 INFO:teuthology.orchestra.run.smithi003.stdout:License : GPLv3+ 2024-09-17T09:13:47.323 INFO:teuthology.orchestra.run.smithi003.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T09:13:47.323 INFO:teuthology.orchestra.run.smithi003.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T09:13:47.323 INFO:teuthology.orchestra.run.smithi003.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T09:13:47.323 INFO:teuthology.orchestra.run.smithi003.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T09:13:47.323 INFO:teuthology.orchestra.run.smithi003.stdout:Packager : builder@centos.org 2024-09-17T09:13:47.323 INFO:teuthology.orchestra.run.smithi003.stdout:Vendor : CentOS 2024-09-17T09:13:47.323 INFO:teuthology.orchestra.run.smithi003.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T09:13:47.324 INFO:teuthology.orchestra.run.smithi003.stdout:Summary : Support tools for GRUB. 2024-09-17T09:13:47.324 INFO:teuthology.orchestra.run.smithi003.stdout:Description : 2024-09-17T09:13:47.324 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:13:47.324 INFO:teuthology.orchestra.run.smithi003.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T09:13:47.324 INFO:teuthology.orchestra.run.smithi003.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T09:13:47.324 INFO:teuthology.orchestra.run.smithi003.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T09:13:47.324 INFO:teuthology.orchestra.run.smithi003.stdout:hardware devices. 2024-09-17T09:13:47.325 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:13:47.325 INFO:teuthology.orchestra.run.smithi003.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T09:13:47.326 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T09:13:47.326 INFO:teuthology.task.kernel:Updating grub on smithi003 to boot 5.14.0-508.el9.x86_64 2024-09-17T09:13:47.326 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T09:13:47.988 INFO:teuthology.orchestra.run.smithi003.stderr:Generating grub configuration file ... 2024-09-17T09:13:49.227 INFO:teuthology.orchestra.run.smithi003.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T09:13:49.256 INFO:teuthology.orchestra.run.smithi003.stderr:done 2024-09-17T09:13:49.258 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2024-09-17T09:13:49.274 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.BZwWW7iyjH 2024-09-17T09:13:49.274 DEBUG:teuthology.orchestra.run.smithi003:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.BZwWW7iyjH 2024-09-17T09:13:49.347 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.BZwWW7iyjH 2024-09-17T09:13:49.486 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.BZwWW7iyjH is 6KB 2024-09-17T09:13:49.535 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.BZwWW7iyjH 2024-09-17T09:13:49.549 DEBUG:teuthology.orchestra.run.smithi003:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T09:13:49.614 INFO:teuthology.orchestra.run.smithi003.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T09:13:49.614 INFO:teuthology.orchestra.run.smithi003.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T09:13:49.614 INFO:teuthology.orchestra.run.smithi003.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T09:13:49.615 INFO:teuthology.orchestra.run.smithi003.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T09:13:49.616 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T09:13:49.766 DEBUG:teuthology.orchestra.run.smithi003:> sudo shutdown -r now 2024-09-17T09:14:01.917 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.77 2024-09-17T09:14:10.925 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-09-17T09:14:10.925 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:14:13.981 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.77 2024-09-17T09:14:15.229 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.183 2024-09-17T09:14:19.769 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-17T09:14:19.770 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-17T09:14:19.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:14:24.234 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-17T09:14:24.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:14:25.984 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-09-17T09:14:25.984 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:14:29.053 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.77 2024-09-17T09:14:38.269 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.3 2024-09-17T09:14:42.625 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.183 2024-09-17T09:14:44.056 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-09-17T09:14:44.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:14:44.438 DEBUG:teuthology.orchestra.run.smithi077:> true 2024-09-17T09:14:44.947 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-09-17T09:14:44.947 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-17T09:14:44.947 DEBUG:teuthology.orchestra.run.smithi077:> uname -r 2024-09-17T09:14:44.982 INFO:teuthology.orchestra.run.smithi077.stdout:5.14.0-508.el9.x86_64 2024-09-17T09:14:44.983 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T09:14:44.983 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T09:14:44.983 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T09:14:45.984 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-17T09:14:45.985 DEBUG:teuthology.orchestra.run.smithi077:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T09:14:46.141 INFO:teuthology.orchestra.run.smithi077.stdout:ttyS1 2024-09-17T09:14:46.390 DEBUG:teuthology.parallel:result is None 2024-09-17T09:14:47.270 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-17T09:14:47.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:14:54.626 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-17T09:14:54.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:14:54.933 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-09-17T09:14:55.311 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-17T09:14:55.311 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-508.el9.x86_64"... 2024-09-17T09:14:55.312 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2024-09-17T09:14:55.326 INFO:teuthology.orchestra.run.smithi183.stdout:5.14.0-508.el9.x86_64 2024-09-17T09:14:55.326 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T09:14:55.326 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T09:14:55.326 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T09:14:56.327 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-17T09:14:56.328 DEBUG:teuthology.orchestra.run.smithi183:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T09:14:56.429 INFO:teuthology.orchestra.run.smithi183.stdout:ttyS1 2024-09-17T09:14:56.464 DEBUG:teuthology.parallel:result is None 2024-09-17T09:15:47.273 DEBUG:teuthology.orchestra.remote:timed out 2024-09-17T09:15:59.275 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-17T09:15:59.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:15:59.667 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-09-17T09:16:00.038 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-17T09:16:00.039 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-17T09:16:00.039 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-09-17T09:16:00.055 INFO:teuthology.orchestra.run.smithi003.stdout:5.14.0-508.el9.x86_64 2024-09-17T09:16:00.056 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T09:16:00.056 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T09:16:00.056 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T09:16:01.057 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-17T09:16:01.057 DEBUG:teuthology.orchestra.run.smithi003:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T09:16:01.139 INFO:teuthology.orchestra.run.smithi003.stdout:ttyS1 2024-09-17T09:16:01.164 DEBUG:teuthology.parallel:result is None 2024-09-17T09:16:01.165 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T09:16:01.172 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T09:16:01.172 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T09:16:01.175 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T09:16:01.178 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T09:16:01.197 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T09:16:01.217 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T09:16:01.341 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T09:16:01.341 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T09:16:01.344 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T09:16:01.346 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T09:16:01.381 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T09:16:01.398 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T09:16:01.398 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T09:16:01.409 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T09:16:01.419 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T09:16:01.443 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T09:16:01.451 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T09:16:01.453 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T09:16:01.454 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T09:16:01.463 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T09:16:01.465 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T09:16:01.466 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T09:16:01.473 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T09:16:01.474 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T09:16:01.499 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T09:16:01.507 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T09:16:01.535 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T09:16:01.545 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T09:16:01.546 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T09:16:01.569 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T09:16:01.576 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T09:16:01.593 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T09:16:01.669 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T09:16:01.715 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T09:16:01.799 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T09:16:01.844 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-17T09:16:01.845 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T09:16:01.910 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T09:16:01.951 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T09:16:02.009 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T09:16:02.091 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T09:16:02.135 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-17T09:16:02.136 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T09:16:02.204 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T09:16:02.245 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T09:16:02.304 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T09:16:02.389 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T09:16:02.435 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-17T09:16:02.435 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T09:16:02.505 DEBUG:teuthology.orchestra.run.smithi003:> sudo service rsyslog restart 2024-09-17T09:16:02.508 DEBUG:teuthology.orchestra.run.smithi077:> sudo service rsyslog restart 2024-09-17T09:16:02.512 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2024-09-17T09:16:02.552 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T09:16:02.557 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T09:16:02.600 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T09:16:03.101 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T09:16:03.109 INFO:teuthology.task.internal:Starting timer... 2024-09-17T09:16:03.109 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T09:16:03.162 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T09:16:03.216 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-17T09:16:03.217 DEBUG:teuthology.orchestra.run.smithi003:> sudo service auditd rotate 2024-09-17T09:16:03.271 INFO:teuthology.orchestra.run.smithi003.stdout:Rotating logs: 2024-09-17T09:16:03.273 DEBUG:teuthology.orchestra.run.smithi077:> sudo service auditd rotate 2024-09-17T09:16:03.359 INFO:teuthology.orchestra.run.smithi077.stdout:Rotating logs: 2024-09-17T09:16:03.361 DEBUG:teuthology.orchestra.run.smithi183:> sudo service auditd rotate 2024-09-17T09:16:03.416 INFO:teuthology.orchestra.run.smithi183.stdout:Rotating logs: 2024-09-17T09:16:03.418 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T09:16:03.418 DEBUG:teuthology.orchestra.run.smithi003:> /usr/sbin/getenforce 2024-09-17T09:16:03.446 INFO:teuthology.orchestra.run.smithi003.stdout:Permissive 2024-09-17T09:16:03.446 DEBUG:teuthology.orchestra.run.smithi077:> /usr/sbin/getenforce 2024-09-17T09:16:03.469 INFO:teuthology.orchestra.run.smithi077.stdout:Permissive 2024-09-17T09:16:03.469 DEBUG:teuthology.orchestra.run.smithi183:> /usr/sbin/getenforce 2024-09-17T09:16:03.494 INFO:teuthology.orchestra.run.smithi183.stdout:Permissive 2024-09-17T09:16:03.494 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi003.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi077.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi183.front.sepia.ceph.com': 'permissive'} 2024-09-17T09:16:03.494 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T09:16:03.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T09:16:03.523 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T09:16:03.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T09:16:03.552 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T09:16:03.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T09:16:03.580 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T09:16:03.581 DEBUG:teuthology.orchestra.run.smithi003:> sudo /usr/sbin/setenforce permissive 2024-09-17T09:16:03.616 DEBUG:teuthology.orchestra.run.smithi077:> sudo /usr/sbin/setenforce permissive 2024-09-17T09:16:03.649 DEBUG:teuthology.orchestra.run.smithi183:> sudo /usr/sbin/setenforce permissive 2024-09-17T09:16:03.683 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T09:16:03.692 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T09:16:03.704 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T09:16:03.735 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi003.front.sepia.ceph.com,smithi077.front.sepia.ceph.com,smithi183.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T09:19:54.791 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi003.front.sepia.ceph.com'), Remote(name='ubuntu@smithi077.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com')] 2024-09-17T09:19:54.793 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-17T09:19:54.794 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:19:54.889 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-09-17T09:19:54.966 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-17T09:19:54.966 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-09-17T09:19:54.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:19:55.042 DEBUG:teuthology.orchestra.run.smithi077:> true 2024-09-17T09:19:55.121 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-09-17T09:19:55.121 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-17T09:19:55.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:19:55.198 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-09-17T09:19:55.283 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-17T09:19:55.284 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T09:19:55.295 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T09:19:55.295 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T09:19:55.296 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T09:19:55.298 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T09:19:55.299 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T09:19:55.301 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T09:19:55.301 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T09:19:55.331 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T09:19:55.331 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T09:19:55.346 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T09:19:55.348 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T09:19:55.358 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T09:19:55.372 INFO:teuthology.orchestra.run.smithi077.stderr:sudo: ntpd: command not found 2024-09-17T09:19:55.374 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T09:19:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: ntpd: command not found 2024-09-17T09:19:55.384 INFO:teuthology.orchestra.run.smithi077.stdout:506 Cannot talk to daemon 2024-09-17T09:19:55.388 INFO:teuthology.orchestra.run.smithi003.stdout:506 Cannot talk to daemon 2024-09-17T09:19:55.399 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T09:19:55.400 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: ntpd: command not found 2024-09-17T09:19:55.403 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T09:19:55.411 INFO:teuthology.orchestra.run.smithi183.stdout:506 Cannot talk to daemon 2024-09-17T09:19:55.413 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T09:19:55.419 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T09:19:55.426 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T09:19:55.441 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T09:19:55.468 INFO:teuthology.orchestra.run.smithi077.stderr:bash: line 1: ntpq: command not found 2024-09-17T09:19:55.472 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T09:19:55.472 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2024-09-17T09:19:55.472 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:19:55.472 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:19:55.472 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:19:55.472 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:19:55.481 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 1: ntpq: command not found 2024-09-17T09:19:55.485 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T09:19:55.485 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2024-09-17T09:19:55.485 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:19:55.485 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:19:55.485 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:19:55.485 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:19:55.495 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-09-17T09:19:55.499 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T09:19:55.499 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-09-17T09:19:55.499 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:19:55.499 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:19:55.499 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:19:55.499 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:19:55.499 INFO:teuthology.run_tasks:Running task pexec... 2024-09-17T09:19:55.509 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-17T09:19:55.509 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi003.front.sepia.ceph.com 2024-09-17T09:19:55.510 DEBUG:teuthology.orchestra.run.smithi003:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-17T09:19:55.510 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi077.front.sepia.ceph.com 2024-09-17T09:19:55.510 DEBUG:teuthology.orchestra.run.smithi077:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-17T09:19:55.510 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi183.front.sepia.ceph.com 2024-09-17T09:19:55.511 DEBUG:teuthology.orchestra.run.smithi183:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-17T09:19:55.858 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-09-17T09:19:55.876 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-09-17T09:19:55.876 INFO:teuthology.orchestra.run.smithi077.stdout: Package Architecture Version Repository Size 2024-09-17T09:19:55.876 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-09-17T09:19:55.876 INFO:teuthology.orchestra.run.smithi077.stdout:Removing: 2024-09-17T09:19:55.876 INFO:teuthology.orchestra.run.smithi077.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-17T09:19:55.876 INFO:teuthology.orchestra.run.smithi077.stdout:Removing unused dependencies: 2024-09-17T09:19:55.876 INFO:teuthology.orchestra.run.smithi077.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-17T09:19:55.876 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-17T09:19:55.876 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-09-17T09:19:55.877 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-09-17T09:19:55.877 INFO:teuthology.orchestra.run.smithi077.stdout:Remove 2 Packages 2024-09-17T09:19:55.877 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-17T09:19:55.877 INFO:teuthology.orchestra.run.smithi077.stdout:Freed space: 5.5 M 2024-09-17T09:19:55.877 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-09-17T09:19:55.880 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-09-17T09:19:55.881 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-09-17T09:19:55.889 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-09-17T09:19:55.891 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-09-17T09:19:55.892 INFO:teuthology.orchestra.run.smithi183.stdout: Package Architecture Version Repository Size 2024-09-17T09:19:55.892 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-09-17T09:19:55.892 INFO:teuthology.orchestra.run.smithi183.stdout:Removing: 2024-09-17T09:19:55.892 INFO:teuthology.orchestra.run.smithi183.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-17T09:19:55.892 INFO:teuthology.orchestra.run.smithi183.stdout:Removing unused dependencies: 2024-09-17T09:19:55.892 INFO:teuthology.orchestra.run.smithi183.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-17T09:19:55.892 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-17T09:19:55.892 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-09-17T09:19:55.892 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-09-17T09:19:55.892 INFO:teuthology.orchestra.run.smithi183.stdout:Remove 2 Packages 2024-09-17T09:19:55.892 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-17T09:19:55.892 INFO:teuthology.orchestra.run.smithi183.stdout:Freed space: 5.5 M 2024-09-17T09:19:55.892 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-09-17T09:19:55.893 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-17T09:19:55.896 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-09-17T09:19:55.896 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-09-17T09:19:55.906 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-17T09:19:55.906 INFO:teuthology.orchestra.run.smithi003.stdout: Package Architecture Version Repository Size 2024-09-17T09:19:55.906 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-17T09:19:55.906 INFO:teuthology.orchestra.run.smithi003.stdout:Removing: 2024-09-17T09:19:55.906 INFO:teuthology.orchestra.run.smithi003.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-17T09:19:55.906 INFO:teuthology.orchestra.run.smithi003.stdout:Removing unused dependencies: 2024-09-17T09:19:55.907 INFO:teuthology.orchestra.run.smithi003.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-17T09:19:55.907 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:19:55.907 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-17T09:19:55.907 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-17T09:19:55.907 INFO:teuthology.orchestra.run.smithi003.stdout:Remove 2 Packages 2024-09-17T09:19:55.907 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:19:55.907 INFO:teuthology.orchestra.run.smithi003.stdout:Freed space: 5.5 M 2024-09-17T09:19:55.907 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-17T09:19:55.911 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-17T09:19:55.911 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-17T09:19:55.942 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-09-17T09:19:55.942 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-09-17T09:19:55.970 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-17T09:19:55.971 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-17T09:19:55.973 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-09-17T09:19:55.973 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-09-17T09:19:56.078 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-09-17T09:19:56.101 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-17T09:19:56.114 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-09-17T09:19:56.165 INFO:teuthology.orchestra.run.smithi077.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-17T09:19:56.176 INFO:teuthology.orchestra.run.smithi003.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-17T09:19:56.188 INFO:teuthology.orchestra.run.smithi183.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-17T09:19:56.221 INFO:teuthology.orchestra.run.smithi077.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T09:19:56.238 INFO:teuthology.orchestra.run.smithi003.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T09:19:56.244 INFO:teuthology.orchestra.run.smithi183.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T09:19:56.701 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T09:19:56.701 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-17T09:19:56.709 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T09:19:56.709 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-17T09:19:56.762 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T09:19:56.763 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-17T09:19:56.937 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-17T09:19:56.937 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-17T09:19:56.937 INFO:teuthology.orchestra.run.smithi183.stdout:Removed: 2024-09-17T09:19:56.937 INFO:teuthology.orchestra.run.smithi183.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T09:19:56.937 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-17T09:19:56.937 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-09-17T09:19:56.945 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-17T09:19:56.945 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:19:56.945 INFO:teuthology.orchestra.run.smithi003.stdout:Removed: 2024-09-17T09:19:56.945 INFO:teuthology.orchestra.run.smithi003.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T09:19:56.945 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:19:56.945 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-17T09:19:57.065 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-17T09:19:57.065 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-17T09:19:57.065 INFO:teuthology.orchestra.run.smithi077.stdout:Removed: 2024-09-17T09:19:57.065 INFO:teuthology.orchestra.run.smithi077.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T09:19:57.065 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-17T09:19:57.065 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-09-17T09:19:57.778 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:00:54 ago on Tue 17 Sep 2024 09:19:03 AM UTC. 2024-09-17T09:19:57.812 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:00:50 ago on Tue 17 Sep 2024 09:19:07 AM UTC. 2024-09-17T09:19:57.916 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-09-17T09:19:57.917 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-09-17T09:19:57.917 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-09-17T09:19:57.917 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-09-17T09:19:57.917 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-09-17T09:19:57.917 INFO:teuthology.orchestra.run.smithi183.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-17T09:19:57.917 INFO:teuthology.orchestra.run.smithi183.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-17T09:19:57.917 INFO:teuthology.orchestra.run.smithi183.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-17T09:19:57.918 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-09-17T09:19:57.918 INFO:teuthology.orchestra.run.smithi183.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-17T09:19:57.918 INFO:teuthology.orchestra.run.smithi183.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-17T09:19:57.918 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-17T09:19:57.918 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-17T09:19:57.918 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-17T09:19:57.918 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-09-17T09:19:57.918 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-09-17T09:19:57.918 INFO:teuthology.orchestra.run.smithi183.stdout:Install 7 Packages 2024-09-17T09:19:57.918 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-17T09:19:57.930 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 5.2 M 2024-09-17T09:19:57.930 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 19 M 2024-09-17T09:19:57.930 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-09-17T09:19:57.937 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:00:54 ago on Tue 17 Sep 2024 09:19:03 AM UTC. 2024-09-17T09:19:57.951 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-17T09:19:57.952 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-17T09:19:57.952 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2024-09-17T09:19:57.952 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-17T09:19:57.952 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-17T09:19:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-17T09:19:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-17T09:19:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-17T09:19:57.953 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-09-17T09:19:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-17T09:19:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-17T09:19:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-17T09:19:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-17T09:19:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:19:57.953 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-17T09:19:57.953 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-17T09:19:57.953 INFO:teuthology.orchestra.run.smithi003.stdout:Install 7 Packages 2024-09-17T09:19:57.953 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:19:57.959 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 5.2 M 2024-09-17T09:19:57.959 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 19 M 2024-09-17T09:19:57.959 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-17T09:19:58.074 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-09-17T09:19:58.074 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-09-17T09:19:58.074 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2024-09-17T09:19:58.074 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-09-17T09:19:58.075 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-09-17T09:19:58.075 INFO:teuthology.orchestra.run.smithi077.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-17T09:19:58.075 INFO:teuthology.orchestra.run.smithi077.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-17T09:19:58.075 INFO:teuthology.orchestra.run.smithi077.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-17T09:19:58.075 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-09-17T09:19:58.075 INFO:teuthology.orchestra.run.smithi077.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-17T09:19:58.075 INFO:teuthology.orchestra.run.smithi077.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-17T09:19:58.075 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-17T09:19:58.075 INFO:teuthology.orchestra.run.smithi077.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-17T09:19:58.075 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-17T09:19:58.075 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-09-17T09:19:58.075 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-09-17T09:19:58.076 INFO:teuthology.orchestra.run.smithi077.stdout:Install 7 Packages 2024-09-17T09:19:58.076 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-17T09:19:58.100 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 5.2 M 2024-09-17T09:19:58.100 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 19 M 2024-09-17T09:19:58.100 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-09-17T09:19:58.217 INFO:teuthology.orchestra.run.smithi183.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 414 kB/s | 44 kB 00:00 2024-09-17T09:19:58.242 INFO:teuthology.orchestra.run.smithi183.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 798 kB/s | 106 kB 00:00 2024-09-17T09:19:58.292 INFO:teuthology.orchestra.run.smithi183.stdout:(3/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.6 MB/s | 84 kB 00:00 2024-09-17T09:19:58.317 INFO:teuthology.orchestra.run.smithi183.stdout:(4/7): python3-configshell-1.1.30-1.el9.noarch. 712 kB/s | 72 kB 00:00 2024-09-17T09:19:58.426 INFO:teuthology.orchestra.run.smithi183.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.8 MB/s | 906 kB 00:00 2024-09-17T09:19:58.493 INFO:teuthology.orchestra.run.smithi183.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 4.1 MB/s | 837 kB 00:00 2024-09-17T09:19:58.635 INFO:teuthology.orchestra.run.smithi077.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 330 kB/s | 44 kB 00:00 2024-09-17T09:19:58.669 INFO:teuthology.orchestra.run.smithi077.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 631 kB/s | 106 kB 00:00 2024-09-17T09:19:58.719 INFO:teuthology.orchestra.run.smithi077.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 858 kB/s | 72 kB 00:00 2024-09-17T09:19:58.744 INFO:teuthology.orchestra.run.smithi077.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.1 MB/s | 84 kB 00:00 2024-09-17T09:19:58.812 INFO:teuthology.orchestra.run.smithi077.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.8 MB/s | 906 kB 00:00 2024-09-17T09:19:58.849 INFO:teuthology.orchestra.run.smithi003.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 75 kB/s | 44 kB 00:00 2024-09-17T09:19:58.913 INFO:teuthology.orchestra.run.smithi077.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 4.3 MB/s | 837 kB 00:00 2024-09-17T09:19:58.916 INFO:teuthology.orchestra.run.smithi003.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 161 kB/s | 106 kB 00:00 2024-09-17T09:19:59.150 INFO:teuthology.orchestra.run.smithi003.stdout:(3/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 1.0 MB/s | 906 kB 00:00 2024-09-17T09:19:59.429 INFO:teuthology.orchestra.run.smithi077.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 4.6 MB/s | 3.2 MB 00:00 2024-09-17T09:19:59.429 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-09-17T09:19:59.429 INFO:teuthology.orchestra.run.smithi077.stdout:Total 3.9 MB/s | 5.2 MB 00:01 2024-09-17T09:19:59.493 INFO:teuthology.orchestra.run.smithi183.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 2.7 MB/s | 3.2 MB 00:01 2024-09-17T09:19:59.494 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-09-17T09:19:59.494 INFO:teuthology.orchestra.run.smithi183.stdout:Total 3.3 MB/s | 5.2 MB 00:01 2024-09-17T09:19:59.525 INFO:teuthology.orchestra.run.smithi003.stdout:(4/7): python3-configshell-1.1.30-1.el9.noarch. 106 kB/s | 72 kB 00:00 2024-09-17T09:19:59.540 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-09-17T09:19:59.555 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-09-17T09:19:59.555 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-09-17T09:19:59.584 INFO:teuthology.orchestra.run.smithi003.stdout:(5/7): python3-kmod-0.9-32.el9.x86_64.rpm 126 kB/s | 84 kB 00:00 2024-09-17T09:19:59.612 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-09-17T09:19:59.628 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-09-17T09:19:59.628 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-09-17T09:19:59.849 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-09-17T09:19:59.849 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-09-17T09:19:59.987 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-09-17T09:19:59.987 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-09-17T09:20:00.042 INFO:teuthology.orchestra.run.smithi003.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 948 kB/s | 837 kB 00:00 2024-09-17T09:20:00.371 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-09-17T09:20:00.463 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-09-17T09:20:00.466 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-17T09:20:00.501 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-17T09:20:00.530 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-17T09:20:00.548 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-17T09:20:00.576 INFO:teuthology.orchestra.run.smithi003.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 3.0 MB/s | 3.2 MB 00:01 2024-09-17T09:20:00.576 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-17T09:20:00.577 INFO:teuthology.orchestra.run.smithi003.stdout:Total 2.0 MB/s | 5.2 MB 00:02 2024-09-17T09:20:00.579 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-17T09:20:00.609 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-17T09:20:00.697 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-17T09:20:00.714 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-17T09:20:00.714 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-17T09:20:00.742 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-17T09:20:00.777 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T09:20:00.826 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-17T09:20:00.857 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T09:20:01.041 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-17T09:20:01.041 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-17T09:20:01.220 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T09:20:01.246 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T09:20:01.303 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T09:20:01.326 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T09:20:01.369 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T09:20:01.433 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T09:20:01.443 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T09:20:01.505 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T09:20:01.542 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-17T09:20:01.646 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-17T09:20:01.686 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-17T09:20:01.716 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-17T09:20:01.947 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-17T09:20:01.981 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T09:20:02.381 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T09:20:02.381 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-17T09:20:02.381 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-17T09:20:02.381 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-17T09:20:02.381 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-17T09:20:02.381 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-17T09:20:02.381 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-17T09:20:02.460 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T09:20:02.460 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-17T09:20:02.460 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-17T09:20:02.460 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-17T09:20:02.460 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-17T09:20:02.460 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-17T09:20:02.460 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-17T09:20:02.473 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T09:20:02.509 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T09:20:02.627 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T09:20:02.698 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T09:20:02.742 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T09:20:02.743 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-17T09:20:02.743 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-09-17T09:20:02.743 INFO:teuthology.orchestra.run.smithi077.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T09:20:02.743 INFO:teuthology.orchestra.run.smithi077.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-17T09:20:02.743 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-17T09:20:02.743 INFO:teuthology.orchestra.run.smithi077.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-17T09:20:02.743 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-17T09:20:02.743 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-09-17T09:20:02.779 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T09:20:02.779 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-17T09:20:02.780 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-09-17T09:20:02.780 INFO:teuthology.orchestra.run.smithi183.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T09:20:02.780 INFO:teuthology.orchestra.run.smithi183.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-17T09:20:02.780 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-17T09:20:02.780 INFO:teuthology.orchestra.run.smithi183.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-17T09:20:02.780 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-17T09:20:02.780 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-09-17T09:20:03.101 DEBUG:teuthology.parallel:result is None 2024-09-17T09:20:03.191 DEBUG:teuthology.parallel:result is None 2024-09-17T09:20:03.713 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T09:20:03.714 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-17T09:20:03.714 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-17T09:20:03.714 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-17T09:20:03.714 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-17T09:20:03.714 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-17T09:20:03.714 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-17T09:20:04.093 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T09:20:04.093 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:20:04.093 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-17T09:20:04.093 INFO:teuthology.orchestra.run.smithi003.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T09:20:04.093 INFO:teuthology.orchestra.run.smithi003.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-17T09:20:04.093 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-17T09:20:04.093 INFO:teuthology.orchestra.run.smithi003.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-17T09:20:04.093 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:20:04.094 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-17T09:20:04.445 DEBUG:teuthology.parallel:result is None 2024-09-17T09:20:04.446 INFO:teuthology.run_tasks:Running task nvme_loop... 2024-09-17T09:20:04.458 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices... 2024-09-17T09:20:04.459 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-17T09:20:04.459 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T09:20:04.475 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T09:20:04.475 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_1 2024-09-17T09:20:04.531 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T09:20:04.531 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T09:20:04.531 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 972 Links: 1 2024-09-17T09:20:04.532 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T09:20:04.532 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T09:20:04.532 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-17 09:20:02.936158879 +0000 2024-09-17T09:20:04.532 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-17 09:20:02.594162844 +0000 2024-09-17T09:20:04.532 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-17 09:20:02.594162844 +0000 2024-09-17T09:20:04.532 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-17 09:20:02.594162844 +0000 2024-09-17T09:20:04.532 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T09:20:04.595 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-17T09:20:04.595 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-17T09:20:04.596 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.00026824 s, 1.9 MB/s 2024-09-17T09:20:04.597 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T09:20:04.654 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_2 2024-09-17T09:20:04.710 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T09:20:04.710 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T09:20:04.710 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 968 Links: 1 2024-09-17T09:20:04.711 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T09:20:04.711 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T09:20:04.711 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-17 09:20:02.936158879 +0000 2024-09-17T09:20:04.711 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-17 09:20:02.593162855 +0000 2024-09-17T09:20:04.711 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-17 09:20:02.593162855 +0000 2024-09-17T09:20:04.711 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-17 09:20:02.593162855 +0000 2024-09-17T09:20:04.711 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T09:20:04.775 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-17T09:20:04.775 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-17T09:20:04.775 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000185658 s, 2.8 MB/s 2024-09-17T09:20:04.777 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T09:20:04.833 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_3 2024-09-17T09:20:04.890 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T09:20:04.890 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T09:20:04.890 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 981 Links: 1 2024-09-17T09:20:04.890 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T09:20:04.890 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T09:20:04.890 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-17 09:20:02.937158868 +0000 2024-09-17T09:20:04.890 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-17 09:20:02.596162821 +0000 2024-09-17T09:20:04.890 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-17 09:20:02.596162821 +0000 2024-09-17T09:20:04.890 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-17 09:20:02.596162821 +0000 2024-09-17T09:20:04.891 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T09:20:04.956 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-17T09:20:04.956 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-17T09:20:04.957 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000220452 s, 2.3 MB/s 2024-09-17T09:20:04.961 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T09:20:05.018 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_4 2024-09-17T09:20:05.074 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T09:20:05.075 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T09:20:05.075 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 980 Links: 1 2024-09-17T09:20:05.075 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T09:20:05.075 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T09:20:05.075 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-17 09:20:02.937158868 +0000 2024-09-17T09:20:05.075 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-17 09:20:02.596162821 +0000 2024-09-17T09:20:05.075 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-17 09:20:02.596162821 +0000 2024-09-17T09:20:05.075 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-17 09:20:02.596162821 +0000 2024-09-17T09:20:05.075 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T09:20:05.140 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-17T09:20:05.140 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-17T09:20:05.140 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000173542 s, 3.0 MB/s 2024-09-17T09:20:05.141 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T09:20:05.199 DEBUG:teuthology.orchestra.run.smithi003:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-17T09:20:05.343 INFO:teuthology.orchestra.run.smithi003.stdout:loop 2024-09-17T09:20:05.344 INFO:tasks.nvme_loop:Connecting nvme_loop smithi003:/dev/vg_nvme/lv_1... 2024-09-17T09:20:05.344 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-17T09:20:05.380 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-17T09:20:05.412 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/vg_nvme/lv_11 2024-09-17T09:20:05.462 INFO:teuthology.orchestra.run.smithi003.stdout:connecting to device: nvme1 2024-09-17T09:20:05.464 INFO:tasks.nvme_loop:Connecting nvme_loop smithi003:/dev/vg_nvme/lv_2... 2024-09-17T09:20:05.464 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-17T09:20:05.539 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-17T09:20:05.571 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/vg_nvme/lv_21 2024-09-17T09:20:05.602 INFO:teuthology.orchestra.run.smithi003.stdout:connecting to device: nvme2 2024-09-17T09:20:05.604 INFO:tasks.nvme_loop:Connecting nvme_loop smithi003:/dev/vg_nvme/lv_3... 2024-09-17T09:20:05.604 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-17T09:20:05.679 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-17T09:20:05.711 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/vg_nvme/lv_31 2024-09-17T09:20:05.747 INFO:teuthology.orchestra.run.smithi003.stdout:connecting to device: nvme3 2024-09-17T09:20:05.749 INFO:tasks.nvme_loop:Connecting nvme_loop smithi003:/dev/vg_nvme/lv_4... 2024-09-17T09:20:05.749 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-17T09:20:05.823 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-17T09:20:05.854 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/vg_nvme/lv_41 2024-09-17T09:20:05.891 INFO:teuthology.orchestra.run.smithi003.stdout:connecting to device: nvme4 2024-09-17T09:20:05.893 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-17T09:20:05.893 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T09:20:05.949 DEBUG:teuthology.orchestra.run.smithi003:> sudo nvme list -o json 2024-09-17T09:20:06.017 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-17T09:20:06.017 INFO:teuthology.orchestra.run.smithi003.stdout: "Devices":[ 2024-09-17T09:20:06.017 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-17T09:20:06.017 INFO:teuthology.orchestra.run.smithi003.stdout: "NameSpace":1, 2024-09-17T09:20:06.017 INFO:teuthology.orchestra.run.smithi003.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-17T09:20:06.017 INFO:teuthology.orchestra.run.smithi003.stdout: "GenericPath":"/dev/ng4n1", 2024-09-17T09:20:06.017 INFO:teuthology.orchestra.run.smithi003.stdout: "Firmware":"5.14.0-5", 2024-09-17T09:20:06.017 INFO:teuthology.orchestra.run.smithi003.stdout: "ModelNumber":"Linux", 2024-09-17T09:20:06.017 INFO:teuthology.orchestra.run.smithi003.stdout: "SerialNumber":"0164f077ff90eda624f2", 2024-09-17T09:20:06.018 INFO:teuthology.orchestra.run.smithi003.stdout: "UsedBytes":95995035648, 2024-09-17T09:20:06.018 INFO:teuthology.orchestra.run.smithi003.stdout: "MaximumLBA":187490304, 2024-09-17T09:20:06.018 INFO:teuthology.orchestra.run.smithi003.stdout: "PhysicalSize":95995035648, 2024-09-17T09:20:06.018 INFO:teuthology.orchestra.run.smithi003.stdout: "SectorSize":512 2024-09-17T09:20:06.018 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-17T09:20:06.018 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-17T09:20:06.018 INFO:teuthology.orchestra.run.smithi003.stdout: "NameSpace":1, 2024-09-17T09:20:06.018 INFO:teuthology.orchestra.run.smithi003.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-17T09:20:06.018 INFO:teuthology.orchestra.run.smithi003.stdout: "GenericPath":"/dev/ng3n1", 2024-09-17T09:20:06.018 INFO:teuthology.orchestra.run.smithi003.stdout: "Firmware":"5.14.0-5", 2024-09-17T09:20:06.018 INFO:teuthology.orchestra.run.smithi003.stdout: "ModelNumber":"Linux", 2024-09-17T09:20:06.018 INFO:teuthology.orchestra.run.smithi003.stdout: "SerialNumber":"778b7aa51d6954a1a7b0", 2024-09-17T09:20:06.019 INFO:teuthology.orchestra.run.smithi003.stdout: "UsedBytes":95995035648, 2024-09-17T09:20:06.019 INFO:teuthology.orchestra.run.smithi003.stdout: "MaximumLBA":187490304, 2024-09-17T09:20:06.019 INFO:teuthology.orchestra.run.smithi003.stdout: "PhysicalSize":95995035648, 2024-09-17T09:20:06.019 INFO:teuthology.orchestra.run.smithi003.stdout: "SectorSize":512 2024-09-17T09:20:06.019 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-17T09:20:06.019 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-17T09:20:06.019 INFO:teuthology.orchestra.run.smithi003.stdout: "NameSpace":1, 2024-09-17T09:20:06.019 INFO:teuthology.orchestra.run.smithi003.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-17T09:20:06.019 INFO:teuthology.orchestra.run.smithi003.stdout: "GenericPath":"/dev/ng2n1", 2024-09-17T09:20:06.019 INFO:teuthology.orchestra.run.smithi003.stdout: "Firmware":"5.14.0-5", 2024-09-17T09:20:06.019 INFO:teuthology.orchestra.run.smithi003.stdout: "ModelNumber":"Linux", 2024-09-17T09:20:06.019 INFO:teuthology.orchestra.run.smithi003.stdout: "SerialNumber":"17f079916edabdad973e", 2024-09-17T09:20:06.020 INFO:teuthology.orchestra.run.smithi003.stdout: "UsedBytes":95995035648, 2024-09-17T09:20:06.020 INFO:teuthology.orchestra.run.smithi003.stdout: "MaximumLBA":187490304, 2024-09-17T09:20:06.020 INFO:teuthology.orchestra.run.smithi003.stdout: "PhysicalSize":95995035648, 2024-09-17T09:20:06.020 INFO:teuthology.orchestra.run.smithi003.stdout: "SectorSize":512 2024-09-17T09:20:06.020 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-17T09:20:06.020 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-17T09:20:06.020 INFO:teuthology.orchestra.run.smithi003.stdout: "NameSpace":1, 2024-09-17T09:20:06.020 INFO:teuthology.orchestra.run.smithi003.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-17T09:20:06.020 INFO:teuthology.orchestra.run.smithi003.stdout: "GenericPath":"/dev/ng1n1", 2024-09-17T09:20:06.020 INFO:teuthology.orchestra.run.smithi003.stdout: "Firmware":"5.14.0-5", 2024-09-17T09:20:06.020 INFO:teuthology.orchestra.run.smithi003.stdout: "ModelNumber":"Linux", 2024-09-17T09:20:06.021 INFO:teuthology.orchestra.run.smithi003.stdout: "SerialNumber":"fadcd9eb6a1368541422", 2024-09-17T09:20:06.021 INFO:teuthology.orchestra.run.smithi003.stdout: "UsedBytes":95995035648, 2024-09-17T09:20:06.021 INFO:teuthology.orchestra.run.smithi003.stdout: "MaximumLBA":187490304, 2024-09-17T09:20:06.021 INFO:teuthology.orchestra.run.smithi003.stdout: "PhysicalSize":95995035648, 2024-09-17T09:20:06.021 INFO:teuthology.orchestra.run.smithi003.stdout: "SectorSize":512 2024-09-17T09:20:06.021 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-17T09:20:06.021 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-17T09:20:06.021 INFO:teuthology.orchestra.run.smithi003.stdout: "NameSpace":1, 2024-09-17T09:20:06.021 INFO:teuthology.orchestra.run.smithi003.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-17T09:20:06.021 INFO:teuthology.orchestra.run.smithi003.stdout: "GenericPath":"/dev/ng0n1", 2024-09-17T09:20:06.021 INFO:teuthology.orchestra.run.smithi003.stdout: "Firmware":"8DV101H0", 2024-09-17T09:20:06.022 INFO:teuthology.orchestra.run.smithi003.stdout: "ModelNumber":"INTEL SSDPEDMD400G4", 2024-09-17T09:20:06.022 INFO:teuthology.orchestra.run.smithi003.stdout: "SerialNumber":"CVFT5330009Z400BGN", 2024-09-17T09:20:06.022 INFO:teuthology.orchestra.run.smithi003.stdout: "UsedBytes":400088457216, 2024-09-17T09:20:06.022 INFO:teuthology.orchestra.run.smithi003.stdout: "MaximumLBA":781422768, 2024-09-17T09:20:06.022 INFO:teuthology.orchestra.run.smithi003.stdout: "PhysicalSize":400088457216, 2024-09-17T09:20:06.022 INFO:teuthology.orchestra.run.smithi003.stdout: "SectorSize":512 2024-09-17T09:20:06.022 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-17T09:20:06.022 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-17T09:20:06.022 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-17T09:20:06.023 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-17T09:20:06.024 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-17T09:20:06.024 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/scratch_devs 2024-09-17T09:20:06.090 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-17T09:20:06.090 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T09:20:06.110 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T09:20:06.110 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_1 2024-09-17T09:20:06.168 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T09:20:06.168 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T09:20:06.169 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 973 Links: 1 2024-09-17T09:20:06.169 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T09:20:06.169 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T09:20:06.169 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-09-17 09:20:01.655690013 +0000 2024-09-17T09:20:06.169 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-09-17 09:20:01.335704267 +0000 2024-09-17T09:20:06.169 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-09-17 09:20:01.335704267 +0000 2024-09-17T09:20:06.169 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-09-17 09:20:01.335704267 +0000 2024-09-17T09:20:06.169 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T09:20:06.237 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-09-17T09:20:06.237 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-09-17T09:20:06.237 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000203908 s, 2.5 MB/s 2024-09-17T09:20:06.238 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T09:20:06.298 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_2 2024-09-17T09:20:06.355 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T09:20:06.355 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T09:20:06.355 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 971 Links: 1 2024-09-17T09:20:06.355 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T09:20:06.355 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T09:20:06.355 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-09-17 09:20:01.655690013 +0000 2024-09-17T09:20:06.355 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-09-17 09:20:01.334705059 +0000 2024-09-17T09:20:06.355 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-09-17 09:20:01.334705059 +0000 2024-09-17T09:20:06.355 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-09-17 09:20:01.334705059 +0000 2024-09-17T09:20:06.355 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T09:20:06.420 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-09-17T09:20:06.421 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-09-17T09:20:06.421 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000223796 s, 2.3 MB/s 2024-09-17T09:20:06.422 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T09:20:06.479 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_3 2024-09-17T09:20:06.536 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T09:20:06.536 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T09:20:06.537 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 976 Links: 1 2024-09-17T09:20:06.537 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T09:20:06.537 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T09:20:06.537 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-09-17 09:20:01.656689993 +0000 2024-09-17T09:20:06.537 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-09-17 09:20:01.335704267 +0000 2024-09-17T09:20:06.537 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-09-17 09:20:01.335704267 +0000 2024-09-17T09:20:06.537 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-09-17 09:20:01.335704267 +0000 2024-09-17T09:20:06.537 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T09:20:06.603 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-09-17T09:20:06.603 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-09-17T09:20:06.603 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000214451 s, 2.4 MB/s 2024-09-17T09:20:06.604 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T09:20:06.662 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_4 2024-09-17T09:20:06.719 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T09:20:06.719 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T09:20:06.719 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 979 Links: 1 2024-09-17T09:20:06.719 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T09:20:06.719 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T09:20:06.720 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-09-17 09:20:01.656689993 +0000 2024-09-17T09:20:06.720 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-09-17 09:20:01.335704267 +0000 2024-09-17T09:20:06.720 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-09-17 09:20:01.335704267 +0000 2024-09-17T09:20:06.720 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-09-17 09:20:01.335704267 +0000 2024-09-17T09:20:06.720 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T09:20:06.783 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-09-17T09:20:06.783 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-09-17T09:20:06.783 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000191797 s, 2.7 MB/s 2024-09-17T09:20:06.785 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T09:20:06.841 DEBUG:teuthology.orchestra.run.smithi077:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-17T09:20:06.977 INFO:teuthology.orchestra.run.smithi077.stdout:loop 2024-09-17T09:20:06.978 INFO:tasks.nvme_loop:Connecting nvme_loop smithi077:/dev/vg_nvme/lv_1... 2024-09-17T09:20:06.978 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-17T09:20:07.011 INFO:teuthology.orchestra.run.smithi077.stdout:1 2024-09-17T09:20:07.040 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/vg_nvme/lv_11 2024-09-17T09:20:07.080 INFO:teuthology.orchestra.run.smithi077.stdout:connecting to device: nvme1 2024-09-17T09:20:07.082 INFO:tasks.nvme_loop:Connecting nvme_loop smithi077:/dev/vg_nvme/lv_2... 2024-09-17T09:20:07.082 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-17T09:20:07.154 INFO:teuthology.orchestra.run.smithi077.stdout:1 2024-09-17T09:20:07.184 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/vg_nvme/lv_21 2024-09-17T09:20:07.219 INFO:teuthology.orchestra.run.smithi077.stdout:connecting to device: nvme2 2024-09-17T09:20:07.221 INFO:tasks.nvme_loop:Connecting nvme_loop smithi077:/dev/vg_nvme/lv_3... 2024-09-17T09:20:07.221 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-17T09:20:07.295 INFO:teuthology.orchestra.run.smithi077.stdout:1 2024-09-17T09:20:07.324 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/vg_nvme/lv_31 2024-09-17T09:20:07.354 INFO:teuthology.orchestra.run.smithi077.stdout:connecting to device: nvme3 2024-09-17T09:20:07.356 INFO:tasks.nvme_loop:Connecting nvme_loop smithi077:/dev/vg_nvme/lv_4... 2024-09-17T09:20:07.356 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-17T09:20:07.429 INFO:teuthology.orchestra.run.smithi077.stdout:1 2024-09-17T09:20:07.458 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/vg_nvme/lv_41 2024-09-17T09:20:07.489 INFO:teuthology.orchestra.run.smithi077.stdout:connecting to device: nvme4 2024-09-17T09:20:07.490 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-17T09:20:07.490 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T09:20:07.548 DEBUG:teuthology.orchestra.run.smithi077:> sudo nvme list -o json 2024-09-17T09:20:07.614 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-17T09:20:07.614 INFO:teuthology.orchestra.run.smithi077.stdout: "Devices":[ 2024-09-17T09:20:07.614 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-17T09:20:07.614 INFO:teuthology.orchestra.run.smithi077.stdout: "NameSpace":1, 2024-09-17T09:20:07.614 INFO:teuthology.orchestra.run.smithi077.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-17T09:20:07.614 INFO:teuthology.orchestra.run.smithi077.stdout: "GenericPath":"/dev/ng4n1", 2024-09-17T09:20:07.614 INFO:teuthology.orchestra.run.smithi077.stdout: "Firmware":"5.14.0-5", 2024-09-17T09:20:07.614 INFO:teuthology.orchestra.run.smithi077.stdout: "ModelNumber":"Linux", 2024-09-17T09:20:07.614 INFO:teuthology.orchestra.run.smithi077.stdout: "SerialNumber":"e99ac0a2a44b99ee9daa", 2024-09-17T09:20:07.614 INFO:teuthology.orchestra.run.smithi077.stdout: "UsedBytes":95995035648, 2024-09-17T09:20:07.614 INFO:teuthology.orchestra.run.smithi077.stdout: "MaximumLBA":187490304, 2024-09-17T09:20:07.615 INFO:teuthology.orchestra.run.smithi077.stdout: "PhysicalSize":95995035648, 2024-09-17T09:20:07.615 INFO:teuthology.orchestra.run.smithi077.stdout: "SectorSize":512 2024-09-17T09:20:07.615 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-17T09:20:07.615 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-17T09:20:07.615 INFO:teuthology.orchestra.run.smithi077.stdout: "NameSpace":1, 2024-09-17T09:20:07.615 INFO:teuthology.orchestra.run.smithi077.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-17T09:20:07.615 INFO:teuthology.orchestra.run.smithi077.stdout: "GenericPath":"/dev/ng3n1", 2024-09-17T09:20:07.615 INFO:teuthology.orchestra.run.smithi077.stdout: "Firmware":"5.14.0-5", 2024-09-17T09:20:07.615 INFO:teuthology.orchestra.run.smithi077.stdout: "ModelNumber":"Linux", 2024-09-17T09:20:07.615 INFO:teuthology.orchestra.run.smithi077.stdout: "SerialNumber":"5ccc6d439af993f9767d", 2024-09-17T09:20:07.615 INFO:teuthology.orchestra.run.smithi077.stdout: "UsedBytes":95995035648, 2024-09-17T09:20:07.615 INFO:teuthology.orchestra.run.smithi077.stdout: "MaximumLBA":187490304, 2024-09-17T09:20:07.616 INFO:teuthology.orchestra.run.smithi077.stdout: "PhysicalSize":95995035648, 2024-09-17T09:20:07.616 INFO:teuthology.orchestra.run.smithi077.stdout: "SectorSize":512 2024-09-17T09:20:07.616 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-17T09:20:07.616 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-17T09:20:07.616 INFO:teuthology.orchestra.run.smithi077.stdout: "NameSpace":1, 2024-09-17T09:20:07.616 INFO:teuthology.orchestra.run.smithi077.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-17T09:20:07.616 INFO:teuthology.orchestra.run.smithi077.stdout: "GenericPath":"/dev/ng2n1", 2024-09-17T09:20:07.616 INFO:teuthology.orchestra.run.smithi077.stdout: "Firmware":"5.14.0-5", 2024-09-17T09:20:07.616 INFO:teuthology.orchestra.run.smithi077.stdout: "ModelNumber":"Linux", 2024-09-17T09:20:07.616 INFO:teuthology.orchestra.run.smithi077.stdout: "SerialNumber":"cb392e9b894c782dc8a0", 2024-09-17T09:20:07.616 INFO:teuthology.orchestra.run.smithi077.stdout: "UsedBytes":95995035648, 2024-09-17T09:20:07.616 INFO:teuthology.orchestra.run.smithi077.stdout: "MaximumLBA":187490304, 2024-09-17T09:20:07.617 INFO:teuthology.orchestra.run.smithi077.stdout: "PhysicalSize":95995035648, 2024-09-17T09:20:07.617 INFO:teuthology.orchestra.run.smithi077.stdout: "SectorSize":512 2024-09-17T09:20:07.617 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-17T09:20:07.617 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-17T09:20:07.617 INFO:teuthology.orchestra.run.smithi077.stdout: "NameSpace":1, 2024-09-17T09:20:07.617 INFO:teuthology.orchestra.run.smithi077.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-17T09:20:07.617 INFO:teuthology.orchestra.run.smithi077.stdout: "GenericPath":"/dev/ng1n1", 2024-09-17T09:20:07.617 INFO:teuthology.orchestra.run.smithi077.stdout: "Firmware":"5.14.0-5", 2024-09-17T09:20:07.617 INFO:teuthology.orchestra.run.smithi077.stdout: "ModelNumber":"Linux", 2024-09-17T09:20:07.617 INFO:teuthology.orchestra.run.smithi077.stdout: "SerialNumber":"0438b117b1dab8fd7af6", 2024-09-17T09:20:07.617 INFO:teuthology.orchestra.run.smithi077.stdout: "UsedBytes":95995035648, 2024-09-17T09:20:07.617 INFO:teuthology.orchestra.run.smithi077.stdout: "MaximumLBA":187490304, 2024-09-17T09:20:07.617 INFO:teuthology.orchestra.run.smithi077.stdout: "PhysicalSize":95995035648, 2024-09-17T09:20:07.618 INFO:teuthology.orchestra.run.smithi077.stdout: "SectorSize":512 2024-09-17T09:20:07.618 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-17T09:20:07.618 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-17T09:20:07.618 INFO:teuthology.orchestra.run.smithi077.stdout: "NameSpace":1, 2024-09-17T09:20:07.618 INFO:teuthology.orchestra.run.smithi077.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-17T09:20:07.618 INFO:teuthology.orchestra.run.smithi077.stdout: "GenericPath":"/dev/ng0n1", 2024-09-17T09:20:07.618 INFO:teuthology.orchestra.run.smithi077.stdout: "Firmware":"8DV101H0", 2024-09-17T09:20:07.618 INFO:teuthology.orchestra.run.smithi077.stdout: "ModelNumber":"INTEL SSDPEDMD400G4", 2024-09-17T09:20:07.618 INFO:teuthology.orchestra.run.smithi077.stdout: "SerialNumber":"PHFT620400ZJ400BGN", 2024-09-17T09:20:07.618 INFO:teuthology.orchestra.run.smithi077.stdout: "UsedBytes":400088457216, 2024-09-17T09:20:07.618 INFO:teuthology.orchestra.run.smithi077.stdout: "MaximumLBA":781422768, 2024-09-17T09:20:07.618 INFO:teuthology.orchestra.run.smithi077.stdout: "PhysicalSize":400088457216, 2024-09-17T09:20:07.618 INFO:teuthology.orchestra.run.smithi077.stdout: "SectorSize":512 2024-09-17T09:20:07.618 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-17T09:20:07.619 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-17T09:20:07.619 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-17T09:20:07.619 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-17T09:20:07.619 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-17T09:20:07.620 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/scratch_devs 2024-09-17T09:20:07.684 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-17T09:20:07.684 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T09:20:07.705 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T09:20:07.705 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2024-09-17T09:20:07.763 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T09:20:07.763 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T09:20:07.763 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 974 Links: 1 2024-09-17T09:20:07.763 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T09:20:07.763 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T09:20:07.763 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-17 09:20:01.728242217 +0000 2024-09-17T09:20:07.763 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-17 09:20:01.414250722 +0000 2024-09-17T09:20:07.764 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-17 09:20:01.414250722 +0000 2024-09-17T09:20:07.764 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-17 09:20:01.414250722 +0000 2024-09-17T09:20:07.764 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T09:20:07.830 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-17T09:20:07.831 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-17T09:20:07.831 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000198964 s, 2.6 MB/s 2024-09-17T09:20:07.832 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T09:20:07.890 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2024-09-17T09:20:07.946 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T09:20:07.946 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T09:20:07.946 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 976 Links: 1 2024-09-17T09:20:07.946 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T09:20:07.946 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T09:20:07.947 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-17 09:20:01.729242190 +0000 2024-09-17T09:20:07.947 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-17 09:20:01.415250695 +0000 2024-09-17T09:20:07.947 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-17 09:20:01.415250695 +0000 2024-09-17T09:20:07.947 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-17 09:20:01.415250695 +0000 2024-09-17T09:20:07.947 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T09:20:08.012 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-17T09:20:08.012 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-17T09:20:08.012 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000186877 s, 2.7 MB/s 2024-09-17T09:20:08.013 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T09:20:08.070 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2024-09-17T09:20:08.125 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T09:20:08.126 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T09:20:08.126 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 984 Links: 1 2024-09-17T09:20:08.126 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T09:20:08.126 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T09:20:08.126 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-17 09:20:01.729242190 +0000 2024-09-17T09:20:08.126 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-17 09:20:01.415250695 +0000 2024-09-17T09:20:08.126 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-17 09:20:01.415250695 +0000 2024-09-17T09:20:08.126 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-17 09:20:01.415250695 +0000 2024-09-17T09:20:08.126 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T09:20:08.190 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-17T09:20:08.190 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-17T09:20:08.190 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000317391 s, 1.6 MB/s 2024-09-17T09:20:08.192 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T09:20:08.248 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2024-09-17T09:20:08.303 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T09:20:08.303 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T09:20:08.303 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 968 Links: 1 2024-09-17T09:20:08.303 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T09:20:08.304 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T09:20:08.304 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-17 09:20:01.729242190 +0000 2024-09-17T09:20:08.304 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-17 09:20:01.414250722 +0000 2024-09-17T09:20:08.304 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-17 09:20:01.414250722 +0000 2024-09-17T09:20:08.304 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-17 09:20:01.414250722 +0000 2024-09-17T09:20:08.304 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T09:20:08.367 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-17T09:20:08.367 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-17T09:20:08.367 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000194976 s, 2.6 MB/s 2024-09-17T09:20:08.368 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T09:20:08.425 DEBUG:teuthology.orchestra.run.smithi183:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-17T09:20:08.564 INFO:teuthology.orchestra.run.smithi183.stdout:loop 2024-09-17T09:20:08.565 INFO:tasks.nvme_loop:Connecting nvme_loop smithi183:/dev/vg_nvme/lv_1... 2024-09-17T09:20:08.566 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-17T09:20:08.598 INFO:teuthology.orchestra.run.smithi183.stdout:1 2024-09-17T09:20:08.627 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/vg_nvme/lv_11 2024-09-17T09:20:08.664 INFO:teuthology.orchestra.run.smithi183.stdout:connecting to device: nvme1 2024-09-17T09:20:08.666 INFO:tasks.nvme_loop:Connecting nvme_loop smithi183:/dev/vg_nvme/lv_2... 2024-09-17T09:20:08.666 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-17T09:20:08.740 INFO:teuthology.orchestra.run.smithi183.stdout:1 2024-09-17T09:20:08.770 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/vg_nvme/lv_21 2024-09-17T09:20:08.799 INFO:teuthology.orchestra.run.smithi183.stdout:connecting to device: nvme2 2024-09-17T09:20:08.801 INFO:tasks.nvme_loop:Connecting nvme_loop smithi183:/dev/vg_nvme/lv_3... 2024-09-17T09:20:08.802 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-17T09:20:08.875 INFO:teuthology.orchestra.run.smithi183.stdout:1 2024-09-17T09:20:08.904 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/vg_nvme/lv_31 2024-09-17T09:20:08.934 INFO:teuthology.orchestra.run.smithi183.stdout:connecting to device: nvme3 2024-09-17T09:20:08.936 INFO:tasks.nvme_loop:Connecting nvme_loop smithi183:/dev/vg_nvme/lv_4... 2024-09-17T09:20:08.936 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-17T09:20:09.009 INFO:teuthology.orchestra.run.smithi183.stdout:1 2024-09-17T09:20:09.038 INFO:teuthology.orchestra.run.smithi183.stdout:/dev/vg_nvme/lv_41 2024-09-17T09:20:09.068 INFO:teuthology.orchestra.run.smithi183.stdout:connecting to device: nvme4 2024-09-17T09:20:09.070 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-17T09:20:09.070 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T09:20:09.126 DEBUG:teuthology.orchestra.run.smithi183:> sudo nvme list -o json 2024-09-17T09:20:09.191 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-17T09:20:09.191 INFO:teuthology.orchestra.run.smithi183.stdout: "Devices":[ 2024-09-17T09:20:09.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-17T09:20:09.192 INFO:teuthology.orchestra.run.smithi183.stdout: "NameSpace":1, 2024-09-17T09:20:09.192 INFO:teuthology.orchestra.run.smithi183.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-17T09:20:09.192 INFO:teuthology.orchestra.run.smithi183.stdout: "GenericPath":"/dev/ng4n1", 2024-09-17T09:20:09.192 INFO:teuthology.orchestra.run.smithi183.stdout: "Firmware":"5.14.0-5", 2024-09-17T09:20:09.192 INFO:teuthology.orchestra.run.smithi183.stdout: "ModelNumber":"Linux", 2024-09-17T09:20:09.192 INFO:teuthology.orchestra.run.smithi183.stdout: "SerialNumber":"c4b0d2b9775d9062e496", 2024-09-17T09:20:09.192 INFO:teuthology.orchestra.run.smithi183.stdout: "UsedBytes":95995035648, 2024-09-17T09:20:09.192 INFO:teuthology.orchestra.run.smithi183.stdout: "MaximumLBA":187490304, 2024-09-17T09:20:09.192 INFO:teuthology.orchestra.run.smithi183.stdout: "PhysicalSize":95995035648, 2024-09-17T09:20:09.192 INFO:teuthology.orchestra.run.smithi183.stdout: "SectorSize":512 2024-09-17T09:20:09.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-17T09:20:09.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-17T09:20:09.193 INFO:teuthology.orchestra.run.smithi183.stdout: "NameSpace":1, 2024-09-17T09:20:09.193 INFO:teuthology.orchestra.run.smithi183.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-17T09:20:09.193 INFO:teuthology.orchestra.run.smithi183.stdout: "GenericPath":"/dev/ng3n1", 2024-09-17T09:20:09.193 INFO:teuthology.orchestra.run.smithi183.stdout: "Firmware":"5.14.0-5", 2024-09-17T09:20:09.193 INFO:teuthology.orchestra.run.smithi183.stdout: "ModelNumber":"Linux", 2024-09-17T09:20:09.193 INFO:teuthology.orchestra.run.smithi183.stdout: "SerialNumber":"b1019c4afc618e8735fc", 2024-09-17T09:20:09.193 INFO:teuthology.orchestra.run.smithi183.stdout: "UsedBytes":95995035648, 2024-09-17T09:20:09.193 INFO:teuthology.orchestra.run.smithi183.stdout: "MaximumLBA":187490304, 2024-09-17T09:20:09.193 INFO:teuthology.orchestra.run.smithi183.stdout: "PhysicalSize":95995035648, 2024-09-17T09:20:09.193 INFO:teuthology.orchestra.run.smithi183.stdout: "SectorSize":512 2024-09-17T09:20:09.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-17T09:20:09.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-17T09:20:09.193 INFO:teuthology.orchestra.run.smithi183.stdout: "NameSpace":1, 2024-09-17T09:20:09.193 INFO:teuthology.orchestra.run.smithi183.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-17T09:20:09.193 INFO:teuthology.orchestra.run.smithi183.stdout: "GenericPath":"/dev/ng2n1", 2024-09-17T09:20:09.194 INFO:teuthology.orchestra.run.smithi183.stdout: "Firmware":"5.14.0-5", 2024-09-17T09:20:09.194 INFO:teuthology.orchestra.run.smithi183.stdout: "ModelNumber":"Linux", 2024-09-17T09:20:09.194 INFO:teuthology.orchestra.run.smithi183.stdout: "SerialNumber":"12079317b919e8de678b", 2024-09-17T09:20:09.194 INFO:teuthology.orchestra.run.smithi183.stdout: "UsedBytes":95995035648, 2024-09-17T09:20:09.194 INFO:teuthology.orchestra.run.smithi183.stdout: "MaximumLBA":187490304, 2024-09-17T09:20:09.194 INFO:teuthology.orchestra.run.smithi183.stdout: "PhysicalSize":95995035648, 2024-09-17T09:20:09.194 INFO:teuthology.orchestra.run.smithi183.stdout: "SectorSize":512 2024-09-17T09:20:09.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-17T09:20:09.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-17T09:20:09.194 INFO:teuthology.orchestra.run.smithi183.stdout: "NameSpace":1, 2024-09-17T09:20:09.194 INFO:teuthology.orchestra.run.smithi183.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-17T09:20:09.194 INFO:teuthology.orchestra.run.smithi183.stdout: "GenericPath":"/dev/ng1n1", 2024-09-17T09:20:09.194 INFO:teuthology.orchestra.run.smithi183.stdout: "Firmware":"5.14.0-5", 2024-09-17T09:20:09.194 INFO:teuthology.orchestra.run.smithi183.stdout: "ModelNumber":"Linux", 2024-09-17T09:20:09.195 INFO:teuthology.orchestra.run.smithi183.stdout: "SerialNumber":"8801d8bb2bb9c79910f3", 2024-09-17T09:20:09.195 INFO:teuthology.orchestra.run.smithi183.stdout: "UsedBytes":95995035648, 2024-09-17T09:20:09.195 INFO:teuthology.orchestra.run.smithi183.stdout: "MaximumLBA":187490304, 2024-09-17T09:20:09.195 INFO:teuthology.orchestra.run.smithi183.stdout: "PhysicalSize":95995035648, 2024-09-17T09:20:09.195 INFO:teuthology.orchestra.run.smithi183.stdout: "SectorSize":512 2024-09-17T09:20:09.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-17T09:20:09.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-17T09:20:09.195 INFO:teuthology.orchestra.run.smithi183.stdout: "NameSpace":1, 2024-09-17T09:20:09.195 INFO:teuthology.orchestra.run.smithi183.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-17T09:20:09.195 INFO:teuthology.orchestra.run.smithi183.stdout: "GenericPath":"/dev/ng0n1", 2024-09-17T09:20:09.195 INFO:teuthology.orchestra.run.smithi183.stdout: "Firmware":"8DV101H0", 2024-09-17T09:20:09.195 INFO:teuthology.orchestra.run.smithi183.stdout: "ModelNumber":"INTEL SSDPEDMD400G4", 2024-09-17T09:20:09.195 INFO:teuthology.orchestra.run.smithi183.stdout: "SerialNumber":"PHFT620500QJ400BGN", 2024-09-17T09:20:09.195 INFO:teuthology.orchestra.run.smithi183.stdout: "UsedBytes":400088457216, 2024-09-17T09:20:09.196 INFO:teuthology.orchestra.run.smithi183.stdout: "MaximumLBA":781422768, 2024-09-17T09:20:09.196 INFO:teuthology.orchestra.run.smithi183.stdout: "PhysicalSize":400088457216, 2024-09-17T09:20:09.196 INFO:teuthology.orchestra.run.smithi183.stdout: "SectorSize":512 2024-09-17T09:20:09.196 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-17T09:20:09.196 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-17T09:20:09.196 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-17T09:20:09.197 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-17T09:20:09.197 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-17T09:20:09.197 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/scratch_devs 2024-09-17T09:20:09.261 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-17T09:20:09.361 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': False}, 'global': {'mon election default strategy': 3}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'CEPHADM_AGENT_DOWN', 'CEPHADM_FAILED_DAEMON'], 'log-only-match': ['CEPHADM_'], 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'} 2024-09-17T09:20:09.361 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T09:20:09.362 INFO:tasks.cephadm:Cluster fsid is 08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:20:09.362 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-17T09:20:09.362 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.3', 'mon.b': '172.21.15.77', 'mon.c': '172.21.15.183'} 2024-09-17T09:20:09.362 INFO:tasks.cephadm:First mon is mon.a on smithi003 2024-09-17T09:20:09.362 INFO:tasks.cephadm:First mgr is y 2024-09-17T09:20:09.362 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-17T09:20:09.363 DEBUG:teuthology.orchestra.run.smithi003:> sudo hostname $(hostname -s) 2024-09-17T09:20:09.394 DEBUG:teuthology.orchestra.run.smithi077:> sudo hostname $(hostname -s) 2024-09-17T09:20:09.424 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname $(hostname -s) 2024-09-17T09:20:09.449 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-17T09:20:09.450 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T09:20:09.593 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56094/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.4-855-g19e2a2be', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.4-855.g19e2a2be'}, 'url': 'https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-16 16:29:21.739533', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'reef', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-17T09:20:09.732 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref reef, sha1 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T09:20:09.733 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/x86_64/flavors/default/cephadm 2024-09-17T09:20:09.734 DEBUG:teuthology.orchestra.run.smithi003:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-17T09:20:10.051 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r--. 1 ubuntu ubuntu 217222 Sep 17 09:20 /home/ubuntu/cephtest/cephadm 2024-09-17T09:20:10.051 DEBUG:teuthology.orchestra.run.smithi077:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-17T09:20:10.380 INFO:teuthology.orchestra.run.smithi077.stdout:-rw-r--r--. 1 ubuntu ubuntu 217222 Sep 17 09:20 /home/ubuntu/cephtest/cephadm 2024-09-17T09:20:10.380 DEBUG:teuthology.orchestra.run.smithi183:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-17T09:20:10.716 INFO:teuthology.orchestra.run.smithi183.stdout:-rw-r--r--. 1 ubuntu ubuntu 217222 Sep 17 09:20 /home/ubuntu/cephtest/cephadm 2024-09-17T09:20:10.717 DEBUG:teuthology.orchestra.run.smithi003:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-17T09:20:10.737 DEBUG:teuthology.orchestra.run.smithi077:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-17T09:20:10.757 DEBUG:teuthology.orchestra.run.smithi183:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-17T09:20:10.787 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 on all hosts... 2024-09-17T09:20:10.787 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 pull 2024-09-17T09:20:10.790 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 pull 2024-09-17T09:20:10.802 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 pull 2024-09-17T09:20:12.288 INFO:teuthology.orchestra.run.smithi077.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T09:20:12.348 INFO:teuthology.orchestra.run.smithi183.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T09:20:12.464 INFO:teuthology.orchestra.run.smithi003.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T09:20:44.748 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-17T09:20:44.748 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable)", 2024-09-17T09:20:44.748 INFO:teuthology.orchestra.run.smithi003.stdout: "image_id": "980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89", 2024-09-17T09:20:44.748 INFO:teuthology.orchestra.run.smithi003.stdout: "repo_digests": [ 2024-09-17T09:20:44.748 INFO:teuthology.orchestra.run.smithi003.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9" 2024-09-17T09:20:44.748 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-17T09:20:44.749 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-17T09:21:04.593 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-17T09:21:04.594 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_version": "ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable)", 2024-09-17T09:21:04.594 INFO:teuthology.orchestra.run.smithi077.stdout: "image_id": "980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89", 2024-09-17T09:21:04.594 INFO:teuthology.orchestra.run.smithi077.stdout: "repo_digests": [ 2024-09-17T09:21:04.594 INFO:teuthology.orchestra.run.smithi077.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9" 2024-09-17T09:21:04.594 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-17T09:21:04.594 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-17T09:21:04.814 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-17T09:21:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_version": "ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable)", 2024-09-17T09:21:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "image_id": "980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89", 2024-09-17T09:21:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "repo_digests": [ 2024-09-17T09:21:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9" 2024-09-17T09:21:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-17T09:21:04.815 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-17T09:21:04.833 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph 2024-09-17T09:21:04.869 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /etc/ceph 2024-09-17T09:21:04.898 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph 2024-09-17T09:21:04.925 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 777 /etc/ceph 2024-09-17T09:21:04.955 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 777 /etc/ceph 2024-09-17T09:21:04.981 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /etc/ceph 2024-09-17T09:21:05.006 INFO:tasks.cephadm:Writing seed config... 2024-09-17T09:21:05.008 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-17T09:21:05.008 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-17T09:21:05.008 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = False 2024-09-17T09:21:05.008 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-09-17T09:21:05.008 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-17T09:21:05.008 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-17T09:21:05.008 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-17T09:21:05.008 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-17T09:21:05.009 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-17T09:21:05.009 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-17T09:21:05.009 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-17T09:21:05.009 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-17T09:21:05.028 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 08f126da-74d6-11ef-bceb-c7b262605968 mon election default strategy = 3 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = False [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-17T09:21:05.029 DEBUG:teuthology.orchestra.run.smithi003:mon.a> sudo journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@mon.a.service 2024-09-17T09:21:05.072 DEBUG:teuthology.orchestra.run.smithi003:mgr.y> sudo journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@mgr.y.service 2024-09-17T09:21:05.115 INFO:tasks.cephadm:Bootstrapping... 2024-09-17T09:21:05.115 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 -v bootstrap --fsid 08f126da-74d6-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.3 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-17T09:21:05.279 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-17T09:21:05.279 INFO:teuthology.orchestra.run.smithi003.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', '-v', 'bootstrap', '--fsid', '08f126da-74d6-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.3', '--skip-admin-label'] 2024-09-17T09:21:05.303 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout 5.2.2 2024-09-17T09:21:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-17T09:21:05.304 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying podman|docker is present... 2024-09-17T09:21:05.328 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout 5.2.2 2024-09-17T09:21:05.328 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying lvm2 is present... 2024-09-17T09:21:05.328 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying time synchronization is in place... 2024-09-17T09:21:05.335 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-17T09:21:05.335 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-17T09:21:05.342 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-17T09:21:05.342 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-09-17T09:21:05.349 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout enabled 2024-09-17T09:21:05.355 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout active 2024-09-17T09:21:05.355 INFO:teuthology.orchestra.run.smithi003.stdout:Unit chronyd.service is enabled and running 2024-09-17T09:21:05.355 INFO:teuthology.orchestra.run.smithi003.stdout:Repeating the final host check... 2024-09-17T09:21:05.377 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout 5.2.2 2024-09-17T09:21:05.377 INFO:teuthology.orchestra.run.smithi003.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-17T09:21:05.377 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl is present 2024-09-17T09:21:05.377 INFO:teuthology.orchestra.run.smithi003.stdout:lvcreate is present 2024-09-17T09:21:05.384 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-17T09:21:05.384 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-17T09:21:05.390 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-17T09:21:05.390 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-09-17T09:21:05.397 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout enabled 2024-09-17T09:21:05.404 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout active 2024-09-17T09:21:05.404 INFO:teuthology.orchestra.run.smithi003.stdout:Unit chronyd.service is enabled and running 2024-09-17T09:21:05.404 INFO:teuthology.orchestra.run.smithi003.stdout:Host looks OK 2024-09-17T09:21:05.404 INFO:teuthology.orchestra.run.smithi003.stdout:Cluster fsid: 08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:21:05.404 INFO:teuthology.orchestra.run.smithi003.stdout:Acquiring lock 140291598418080 on /run/cephadm/08f126da-74d6-11ef-bceb-c7b262605968.lock 2024-09-17T09:21:05.404 INFO:teuthology.orchestra.run.smithi003.stdout:Lock 140291598418080 acquired on /run/cephadm/08f126da-74d6-11ef-bceb-c7b262605968.lock 2024-09-17T09:21:05.405 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying IP 172.21.15.3 port 3300 ... 2024-09-17T09:21:05.405 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying IP 172.21.15.3 port 6789 ... 2024-09-17T09:21:05.405 INFO:teuthology.orchestra.run.smithi003.stdout:Base mon IP(s) is [172.21.15.3:3300, 172.21.15.3:6789], mon addrv is [v2:172.21.15.3:3300,v1:172.21.15.3:6789] 2024-09-17T09:21:05.408 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.3 metric 100 2024-09-17T09:21:05.408 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.3 metric 100 2024-09-17T09:21:05.411 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-17T09:21:05.411 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-17T09:21:05.411 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1794sec hoplimit 64 pref medium 2024-09-17T09:21:05.413 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-17T09:21:05.413 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-17T09:21:05.414 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-17T09:21:05.414 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-17T09:21:05.414 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-17T09:21:05.414 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-17T09:21:05.414 INFO:teuthology.orchestra.run.smithi003.stdout:Mon IP `172.21.15.3` is in CIDR network `172.21.0.0/20` 2024-09-17T09:21:05.414 INFO:teuthology.orchestra.run.smithi003.stdout:Mon IP `172.21.15.3` is in CIDR network `172.21.0.0/20` 2024-09-17T09:21:05.414 INFO:teuthology.orchestra.run.smithi003.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-17T09:21:05.415 INFO:teuthology.orchestra.run.smithi003.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-17T09:21:05.415 INFO:teuthology.orchestra.run.smithi003.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T09:21:08.732 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 2024-09-17T09:21:08.732 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T09:21:08.732 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Getting image source signatures 2024-09-17T09:21:08.733 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Copying blob sha256:d8a707c21dc06a64315160cb2d8e524fa91943a344731b2a930ee3a716bebbdd 2024-09-17T09:21:08.733 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-17T09:21:08.733 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Copying config sha256:980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 2024-09-17T09:21:08.733 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-17T09:21:14.960 INFO:teuthology.orchestra.run.smithi003.stdout:ceph: stdout ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable) 2024-09-17T09:21:14.960 INFO:teuthology.orchestra.run.smithi003.stdout:Ceph version: ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable) 2024-09-17T09:21:14.960 INFO:teuthology.orchestra.run.smithi003.stdout:Extracting ceph user uid/gid from container image... 2024-09-17T09:21:15.923 INFO:teuthology.orchestra.run.smithi003.stdout:stat: stdout 167 167 2024-09-17T09:21:15.923 INFO:teuthology.orchestra.run.smithi003.stdout:Creating initial keys... 2024-09-17T09:21:16.996 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-authtool: stdout AQAMSulmUZlhHBAAuemBqVn+RK7lqptvmJNfLA== 2024-09-17T09:21:17.850 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-authtool: stdout AQANSulmQ7KtFBAAKLng11bXtdtrbUkabKPlYQ== 2024-09-17T09:21:18.839 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-authtool: stdout AQAOSulmwDsECxAA0982xvOkxNjUFXR8ixZFDw== 2024-09-17T09:21:18.839 INFO:teuthology.orchestra.run.smithi003.stdout:Creating initial monmap... 2024-09-17T09:21:19.803 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-17T09:21:19.804 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-09-17T09:21:19.804 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:21:19.804 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-17T09:21:19.804 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool for a [v2:172.21.15.3:3300,v1:172.21.15.3:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-17T09:21:19.804 INFO:teuthology.orchestra.run.smithi003.stdout:setting min_mon_release = pacific 2024-09-17T09:21:19.804 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: set fsid to 08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:21:19.804 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-17T09:21:19.804 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:21:19.804 INFO:teuthology.orchestra.run.smithi003.stdout:Creating mon... 2024-09-17T09:21:21.573 INFO:teuthology.orchestra.run.smithi003.stdout:create mon.a on 2024-09-17T09:21:21.953 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-17T09:21:22.136 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-08f126da-74d6-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-08f126da-74d6-11ef-bceb-c7b262605968.target. 2024-09-17T09:21:22.136 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-08f126da-74d6-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-08f126da-74d6-11ef-bceb-c7b262605968.target. 2024-09-17T09:21:22.367 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-08f126da-74d6-11ef-bceb-c7b262605968@mon.a 2024-09-17T09:21:22.367 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to reset failed state of unit ceph-08f126da-74d6-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-08f126da-74d6-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-17T09:21:22.545 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-08f126da-74d6-11ef-bceb-c7b262605968.target.wants/ceph-08f126da-74d6-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-08f126da-74d6-11ef-bceb-c7b262605968@.service. 2024-09-17T09:21:23.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:22 smithi003 ceph-mon[25438]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-17T09:21:23.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:23 smithi003 bash[25401]: f9ed82f7637204819a98bf8406e7d23df549c8b7473cf590fc444edecbeb5ac8 2024-09-17T09:21:23.062 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T09:21:23.062 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-09-17T09:21:23.070 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T09:21:23.070 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-09-17T09:21:23.070 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-09-17T09:21:23.070 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to enable service . firewalld.service is not available 2024-09-17T09:21:23.070 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mon to start... 2024-09-17T09:21:23.070 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mon... 2024-09-17T09:21:23.322 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:23 smithi003 systemd[1]: Started Ceph mon.a for 08f126da-74d6-11ef-bceb-c7b262605968. 2024-09-17T09:21:24.084 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout cluster: 2024-09-17T09:21:24.084 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout id: 08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:21:24.084 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-17T09:21:24.084 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-17T09:21:24.084 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout services: 2024-09-17T09:21:24.084 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.619003s) 2024-09-17T09:21:24.084 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-17T09:21:24.084 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-17T09:21:24.084 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-17T09:21:24.085 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout data: 2024-09-17T09:21:24.085 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-17T09:21:24.085 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-17T09:21:24.085 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-17T09:21:24.085 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout pgs: 2024-09-17T09:21:24.085 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-17T09:21:24.085 INFO:teuthology.orchestra.run.smithi003.stdout:mon is available 2024-09-17T09:21:24.085 INFO:teuthology.orchestra.run.smithi003.stdout:Assimilating anything we can from ceph.conf... 2024-09-17T09:21:24.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:23 smithi003 ceph-mon[25438]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-17T09:21:24.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:23 smithi003 ceph-mon[25438]: monmap e1: 1 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T09:21:24.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:23 smithi003 ceph-mon[25438]: fsmap 2024-09-17T09:21:24.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:23 smithi003 ceph-mon[25438]: osdmap e1: 0 total, 0 up, 0 in 2024-09-17T09:21:24.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:23 smithi003 ceph-mon[25438]: mgrmap e1: no daemons active 2024-09-17T09:21:24.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:23 smithi003 ceph-mon[25438]: from='client.? 172.21.15.3:0/691011639' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-17T09:21:25.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:25 smithi003 ceph-mon[25438]: from='client.? 172.21.15.3:0/3071280116' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-17T09:21:25.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:25 smithi003 ceph-mon[25438]: from='client.? 172.21.15.3:0/3071280116' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-17T09:21:25.181 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-17T09:21:25.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [global] 2024-09-17T09:21:25.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout fsid = 08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:21:25.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.3:3300,v1:172.21.15.3:6789] 2024-09-17T09:21:25.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-17T09:21:25.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-17T09:21:25.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-17T09:21:25.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-17T09:21:25.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-17T09:21:25.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-17T09:21:25.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr/cephadm/use_agent = False 2024-09-17T09:21:25.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-17T09:21:25.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-17T09:21:25.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [osd] 2024-09-17T09:21:25.183 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-17T09:21:25.183 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-17T09:21:25.183 INFO:teuthology.orchestra.run.smithi003.stdout:Generating new minimal ceph.conf... 2024-09-17T09:21:26.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:26 smithi003 ceph-mon[25438]: from='client.? 172.21.15.3:0/43703731' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:21:26.394 INFO:teuthology.orchestra.run.smithi003.stdout:Restarting the monitor... 2024-09-17T09:21:26.664 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:26 smithi003 systemd[1]: Stopping Ceph mon.a for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:21:27.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:26 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-a[25415]: 2024-09-17T09:21:26.662+0000 7f0c43da4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:21:27.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:26 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-a[25415]: 2024-09-17T09:21:26.662+0000 7f0c43da4640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-17T09:21:27.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:26 smithi003 podman[25754]: 2024-09-17 09:21:26.785728215 +0000 UTC m=+0.225447290 container died f9ed82f7637204819a98bf8406e7d23df549c8b7473cf590fc444edecbeb5ac8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-a, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-17T09:21:27.403 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 podman[25754]: 2024-09-17 09:21:27.134994981 +0000 UTC m=+0.574714045 container cleanup f9ed82f7637204819a98bf8406e7d23df549c8b7473cf590fc444edecbeb5ac8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-17T09:21:27.403 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 bash[25754]: ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-a 2024-09-17T09:21:27.753 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 podman[25772]: 2024-09-17 09:21:27.402578982 +0000 UTC m=+0.611036056 container remove f9ed82f7637204819a98bf8406e7d23df549c8b7473cf590fc444edecbeb5ac8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-17T09:21:27.754 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 systemd[1]: ceph-08f126da-74d6-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-17T09:21:27.754 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 systemd[1]: Stopped Ceph mon.a for 08f126da-74d6-11ef-bceb-c7b262605968. 2024-09-17T09:21:27.754 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 systemd[1]: Starting Ceph mon.a for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:21:28.006 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 podman[25870]: 2024-09-17 09:21:27.753284574 +0000 UTC m=+0.095141334 container create 9b15ead85423fee8d984a43bc8035cfa2a7109fc37a0aabd624aa117ef98a0e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-17T09:21:28.006 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 podman[25870]: 2024-09-17 09:21:27.674063511 +0000 UTC m=+0.015920286 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T09:21:28.006 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 podman[25870]: 2024-09-17 09:21:27.922123236 +0000 UTC m=+0.263980000 container init 9b15ead85423fee8d984a43bc8035cfa2a7109fc37a0aabd624aa117ef98a0e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True) 2024-09-17T09:21:28.006 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 podman[25870]: 2024-09-17 09:21:27.927200823 +0000 UTC m=+0.269057584 container start 9b15ead85423fee8d984a43bc8035cfa2a7109fc37a0aabd624aa117ef98a0e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-17T09:21:28.006 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: set uid:gid to 167:167 (ceph:ceph) 2024-09-17T09:21:28.006 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable), process ceph-mon, pid 7 2024-09-17T09:21:28.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: pidfile_write: ignore empty --pid-file 2024-09-17T09:21:28.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: load: jerasure load: lrc 2024-09-17T09:21:28.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: RocksDB version: 7.9.2 2024-09-17T09:21:28.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Git sha 0 2024-09-17T09:21:28.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Compile date 2024-09-16 15:15:13 2024-09-17T09:21:28.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: DB SUMMARY 2024-09-17T09:21:28.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: DB Session ID: BTHTZXSK4EC6RIMXXESO 2024-09-17T09:21:28.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: CURRENT file: CURRENT 2024-09-17T09:21:28.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: IDENTITY file: IDENTITY 2024-09-17T09:21:28.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-17T09:21:28.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-17T09:21:28.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 83700 ; 2024-09-17T09:21:28.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.error_if_exists: 0 2024-09-17T09:21:28.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.create_if_missing: 0 2024-09-17T09:21:28.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.paranoid_checks: 1 2024-09-17T09:21:28.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-17T09:21:28.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-17T09:21:28.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-17T09:21:28.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.env: 0x564db0639bc0 2024-09-17T09:21:28.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.fs: PosixFileSystem 2024-09-17T09:21:28.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.info_log: 0x564db132cbe0 2024-09-17T09:21:28.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_file_opening_threads: 16 2024-09-17T09:21:28.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.statistics: (nil) 2024-09-17T09:21:28.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.use_fsync: 0 2024-09-17T09:21:28.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_log_file_size: 0 2024-09-17T09:21:28.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-17T09:21:28.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-17T09:21:28.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.keep_log_file_num: 1000 2024-09-17T09:21:28.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.recycle_log_file_num: 0 2024-09-17T09:21:28.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.allow_fallocate: 1 2024-09-17T09:21:28.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.allow_mmap_reads: 0 2024-09-17T09:21:28.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.allow_mmap_writes: 0 2024-09-17T09:21:28.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.use_direct_reads: 0 2024-09-17T09:21:28.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-17T09:21:28.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.create_missing_column_families: 0 2024-09-17T09:21:28.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.db_log_dir: 2024-09-17T09:21:28.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.wal_dir: 2024-09-17T09:21:28.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-17T09:21:28.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-17T09:21:28.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-17T09:21:28.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-17T09:21:28.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-17T09:21:28.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-17T09:21:28.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.advise_random_on_open: 1 2024-09-17T09:21:28.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.db_write_buffer_size: 0 2024-09-17T09:21:28.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.write_buffer_manager: 0x564db133a320 2024-09-17T09:21:28.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-17T09:21:28.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-17T09:21:28.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-17T09:21:28.011 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.rate_limiter: (nil) 2024-09-17T09:21:28.011 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-17T09:21:28.011 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.wal_recovery_mode: 2 2024-09-17T09:21:28.012 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.enable_thread_tracking: 0 2024-09-17T09:21:28.012 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.enable_pipelined_write: 0 2024-09-17T09:21:28.012 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.unordered_write: 0 2024-09-17T09:21:28.012 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-17T09:21:28.012 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-17T09:21:28.013 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-17T09:21:28.013 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-17T09:21:28.013 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.row_cache: None 2024-09-17T09:21:28.013 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.wal_filter: None 2024-09-17T09:21:28.013 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-17T09:21:28.013 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.allow_ingest_behind: 0 2024-09-17T09:21:28.013 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.two_write_queues: 0 2024-09-17T09:21:28.013 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.manual_wal_flush: 0 2024-09-17T09:21:28.013 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.wal_compression: 0 2024-09-17T09:21:28.013 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.atomic_flush: 0 2024-09-17T09:21:28.013 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-17T09:21:28.013 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-17T09:21:28.013 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-17T09:21:28.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.log_readahead_size: 0 2024-09-17T09:21:28.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-17T09:21:28.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.best_efforts_recovery: 0 2024-09-17T09:21:28.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-17T09:21:28.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-17T09:21:28.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.allow_data_in_errors: 0 2024-09-17T09:21:28.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.db_host_id: __hostname__ 2024-09-17T09:21:28.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-17T09:21:28.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_background_jobs: 2 2024-09-17T09:21:28.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_background_compactions: -1 2024-09-17T09:21:28.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_subcompactions: 1 2024-09-17T09:21:28.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-17T09:21:28.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-17T09:21:28.015 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-17T09:21:28.015 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_total_wal_size: 0 2024-09-17T09:21:28.015 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-17T09:21:28.015 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-17T09:21:28.015 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-17T09:21:28.015 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-17T09:21:28.015 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_open_files: -1 2024-09-17T09:21:28.015 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.bytes_per_sync: 0 2024-09-17T09:21:28.015 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-17T09:21:28.015 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-17T09:21:28.015 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compaction_readahead_size: 0 2024-09-17T09:21:28.015 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_background_flushes: -1 2024-09-17T09:21:28.015 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Compression algorithms supported: 2024-09-17T09:21:28.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: kZSTD supported: 0 2024-09-17T09:21:28.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: kXpressCompression supported: 0 2024-09-17T09:21:28.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: kBZip2Compression supported: 0 2024-09-17T09:21:28.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-17T09:21:28.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: kLZ4Compression supported: 1 2024-09-17T09:21:28.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: kZlibCompression supported: 1 2024-09-17T09:21:28.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: kLZ4HCCompression supported: 1 2024-09-17T09:21:28.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: kSnappyCompression supported: 1 2024-09-17T09:21:28.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-17T09:21:28.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-17T09:21:28.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-17T09:21:28.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-17T09:21:28.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-17T09:21:28.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.merge_operator: 2024-09-17T09:21:28.017 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compaction_filter: None 2024-09-17T09:21:28.017 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compaction_filter_factory: None 2024-09-17T09:21:28.017 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.sst_partitioner_factory: None 2024-09-17T09:21:28.017 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-17T09:21:28.017 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-17T09:21:28.017 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564db132c7e0) 2024-09-17T09:21:28.017 INFO:journalctl@ceph.mon.a.smithi003.stdout: cache_index_and_filter_blocks: 1 2024-09-17T09:21:28.017 INFO:journalctl@ceph.mon.a.smithi003.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-17T09:21:28.017 INFO:journalctl@ceph.mon.a.smithi003.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-17T09:21:28.017 INFO:journalctl@ceph.mon.a.smithi003.stdout: pin_top_level_index_and_filter: 1 2024-09-17T09:21:28.017 INFO:journalctl@ceph.mon.a.smithi003.stdout: index_type: 0 2024-09-17T09:21:28.017 INFO:journalctl@ceph.mon.a.smithi003.stdout: data_block_index_type: 0 2024-09-17T09:21:28.017 INFO:journalctl@ceph.mon.a.smithi003.stdout: index_shortening: 1 2024-09-17T09:21:28.018 INFO:journalctl@ceph.mon.a.smithi003.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-17T09:21:28.018 INFO:journalctl@ceph.mon.a.smithi003.stdout: checksum: 4 2024-09-17T09:21:28.018 INFO:journalctl@ceph.mon.a.smithi003.stdout: no_block_cache: 0 2024-09-17T09:21:28.018 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache: 0x564db1321350 2024-09-17T09:21:28.018 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache_name: BinnedLRUCache 2024-09-17T09:21:28.018 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache_options: 2024-09-17T09:21:28.018 INFO:journalctl@ceph.mon.a.smithi003.stdout: capacity : 536870912 2024-09-17T09:21:28.018 INFO:journalctl@ceph.mon.a.smithi003.stdout: num_shard_bits : 4 2024-09-17T09:21:28.018 INFO:journalctl@ceph.mon.a.smithi003.stdout: strict_capacity_limit : 0 2024-09-17T09:21:28.018 INFO:journalctl@ceph.mon.a.smithi003.stdout: high_pri_pool_ratio: 0.000 2024-09-17T09:21:28.018 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache_compressed: (nil) 2024-09-17T09:21:28.018 INFO:journalctl@ceph.mon.a.smithi003.stdout: persistent_cache: (nil) 2024-09-17T09:21:28.018 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_size: 4096 2024-09-17T09:21:28.019 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_size_deviation: 10 2024-09-17T09:21:28.019 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_restart_interval: 16 2024-09-17T09:21:28.019 INFO:journalctl@ceph.mon.a.smithi003.stdout: index_block_restart_interval: 1 2024-09-17T09:21:28.019 INFO:journalctl@ceph.mon.a.smithi003.stdout: metadata_block_size: 4096 2024-09-17T09:21:28.019 INFO:journalctl@ceph.mon.a.smithi003.stdout: partition_filters: 0 2024-09-17T09:21:28.019 INFO:journalctl@ceph.mon.a.smithi003.stdout: use_delta_encoding: 1 2024-09-17T09:21:28.019 INFO:journalctl@ceph.mon.a.smithi003.stdout: filter_policy: bloomfilter 2024-09-17T09:21:28.019 INFO:journalctl@ceph.mon.a.smithi003.stdout: whole_key_filtering: 1 2024-09-17T09:21:28.019 INFO:journalctl@ceph.mon.a.smithi003.stdout: verify_compression: 0 2024-09-17T09:21:28.019 INFO:journalctl@ceph.mon.a.smithi003.stdout: read_amp_bytes_per_bit: 0 2024-09-17T09:21:28.019 INFO:journalctl@ceph.mon.a.smithi003.stdout: format_version: 5 2024-09-17T09:21:28.019 INFO:journalctl@ceph.mon.a.smithi003.stdout: enable_index_compression: 1 2024-09-17T09:21:28.019 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_align: 0 2024-09-17T09:21:28.020 INFO:journalctl@ceph.mon.a.smithi003.stdout: max_auto_readahead_size: 262144 2024-09-17T09:21:28.020 INFO:journalctl@ceph.mon.a.smithi003.stdout: prepopulate_block_cache: 0 2024-09-17T09:21:28.020 INFO:journalctl@ceph.mon.a.smithi003.stdout: initial_auto_readahead_size: 8192 2024-09-17T09:21:28.020 INFO:journalctl@ceph.mon.a.smithi003.stdout: num_file_reads_for_auto_readahead: 2 2024-09-17T09:21:28.020 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.write_buffer_size: 33554432 2024-09-17T09:21:28.020 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_write_buffer_number: 2 2024-09-17T09:21:28.020 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compression: NoCompression 2024-09-17T09:21:28.020 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.bottommost_compression: Disabled 2024-09-17T09:21:28.020 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.prefix_extractor: nullptr 2024-09-17T09:21:28.020 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-17T09:21:28.020 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.num_levels: 7 2024-09-17T09:21:28.020 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-17T09:21:28.021 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-17T09:21:28.021 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-17T09:21:28.021 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-17T09:21:28.021 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-17T09:21:28.021 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-17T09:21:28.021 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-17T09:21:28.021 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-17T09:21:28.021 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-17T09:21:28.021 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-17T09:21:28.021 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-17T09:21:28.021 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-17T09:21:28.021 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-17T09:21:28.021 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compression_opts.level: 32767 2024-09-17T09:21:28.022 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compression_opts.strategy: 0 2024-09-17T09:21:28.022 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-17T09:21:28.022 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-17T09:21:28.022 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-17T09:21:28.022 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-17T09:21:28.022 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compression_opts.enabled: false 2024-09-17T09:21:28.022 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-17T09:21:28.022 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-17T09:21:28.022 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-17T09:21:28.022 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-17T09:21:28.022 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.target_file_size_base: 67108864 2024-09-17T09:21:28.022 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-17T09:21:28.022 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-17T09:21:28.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-17T09:21:28.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-17T09:21:28.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-17T09:21:28.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-17T09:21:28.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-17T09:21:28.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-17T09:21:28.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-17T09:21:28.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-17T09:21:28.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-17T09:21:28.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-17T09:21:28.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-17T09:21:28.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-17T09:21:28.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.arena_block_size: 1048576 2024-09-17T09:21:28.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-17T09:21:28.024 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-17T09:21:28.024 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.disable_auto_compactions: 0 2024-09-17T09:21:28.024 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-17T09:21:28.024 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-17T09:21:28.024 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-17T09:21:28.024 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-17T09:21:28.024 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-17T09:21:28.024 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-17T09:21:28.024 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-17T09:21:28.024 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-17T09:21:28.024 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-17T09:21:28.024 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-17T09:21:28.024 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-17T09:21:28.025 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.inplace_update_support: 0 2024-09-17T09:21:28.025 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-17T09:21:28.025 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-17T09:21:28.025 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-17T09:21:28.025 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-17T09:21:28.025 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.bloom_locality: 0 2024-09-17T09:21:28.025 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.max_successive_merges: 0 2024-09-17T09:21:28.025 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-17T09:21:28.025 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.paranoid_file_checks: 0 2024-09-17T09:21:28.025 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.force_consistency_checks: 1 2024-09-17T09:21:28.025 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.report_bg_io_stats: 0 2024-09-17T09:21:28.025 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.ttl: 2592000 2024-09-17T09:21:28.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-17T09:21:28.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-17T09:21:28.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-17T09:21:28.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.enable_blob_files: false 2024-09-17T09:21:28.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.min_blob_size: 0 2024-09-17T09:21:28.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.blob_file_size: 268435456 2024-09-17T09:21:28.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-17T09:21:28.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-17T09:21:28.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-17T09:21:28.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-17T09:21:28.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-17T09:21:28.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.blob_file_starting_level: 0 2024-09-17T09:21:28.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-17T09:21:28.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-17T09:21:28.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-17T09:21:28.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e3edbbdc-eebe-460e-8c0a-f20c524bbb81 2024-09-17T09:21:28.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726564887954209, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-17T09:21:28.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-17T09:21:28.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726564887955117, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 80473, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 250, "table_properties": {"data_size": 78634, "index_size": 223, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 9896, "raw_average_key_size": 47, "raw_value_size": 72942, "raw_average_value_size": 349, "num_data_blocks": 10, "num_entries": 209, "num_filter_entries": 209, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726564887, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e3edbbdc-eebe-460e-8c0a-f20c524bbb81", "db_session_id": "BTHTZXSK4EC6RIMXXESO", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-17T09:21:28.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726564887955226, "job": 1, "event": "recovery_finished"} 2024-09-17T09:21:28.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-17T09:21:28.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-17T09:21:28.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564db134ce00 2024-09-17T09:21:28.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: DB pointer 0x564db1440000 2024-09-17T09:21:28.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-17T09:21:28.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-17T09:21:28.027 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** DB Stats ** 2024-09-17T09:21:28.028 INFO:journalctl@ceph.mon.a.smithi003.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T09:21:28.028 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-17T09:21:28.028 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T09:21:28.028 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T09:21:28.028 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-17T09:21:28.028 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T09:21:28.028 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T09:21:28.028 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-09-17T09:21:28.028 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** Compaction Stats [default] ** 2024-09-17T09:21:28.028 INFO:journalctl@ceph.mon.a.smithi003.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T09:21:28.028 INFO:journalctl@ceph.mon.a.smithi003.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-17T09:21:28.028 INFO:journalctl@ceph.mon.a.smithi003.stdout: L0 2/0 80.45 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T09:21:28.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: Sum 2/0 80.45 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T09:21:28.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T09:21:28.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-09-17T09:21:28.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** Compaction Stats [default] ** 2024-09-17T09:21:28.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T09:21:28.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-17T09:21:28.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 134.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T09:21:28.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-09-17T09:21:28.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-17T09:21:28.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-09-17T09:21:28.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T09:21:28.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-17T09:21:28.030 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-17T09:21:28.030 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-17T09:21:28.030 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-17T09:21:28.030 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-17T09:21:28.030 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative compaction: 0.00 GB write, 13.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T09:21:28.030 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval compaction: 0.00 GB write, 13.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T09:21:28.030 INFO:journalctl@ceph.mon.a.smithi003.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-17T09:21:28.030 INFO:journalctl@ceph.mon.a.smithi003.stdout: Block cache BinnedLRUCache@0x564db1321350#7 capacity: 512.00 MB usage: 1.17 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-17T09:21:28.030 INFO:journalctl@ceph.mon.a.smithi003.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.41 KB,7.7486e-05%) Misc(1,0.00 KB,0%) 2024-09-17T09:21:28.030 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-09-17T09:21:28.030 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-17T09:21:28.030 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: starting mon.a rank 0 at public addrs [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] at bind addrs [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:21:28.030 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: mon.a@-1(???) e1 preinit fsid 08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:21:28.030 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: mon.a@-1(???).mds e1 new map 2024-09-17T09:21:28.030 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: mon.a@-1(???).mds e1 print_map 2024-09-17T09:21:28.031 INFO:journalctl@ceph.mon.a.smithi003.stdout: e1 2024-09-17T09:21:28.031 INFO:journalctl@ceph.mon.a.smithi003.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T09:21:28.031 INFO:journalctl@ceph.mon.a.smithi003.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T09:21:28.031 INFO:journalctl@ceph.mon.a.smithi003.stdout: legacy client fscid: -1 2024-09-17T09:21:28.031 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-09-17T09:21:28.031 INFO:journalctl@ceph.mon.a.smithi003.stdout: No filesystems configured 2024-09-17T09:21:28.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-17T09:21:28.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T09:21:28.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T09:21:28.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T09:21:28.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:27 smithi003 ceph-mon[25908]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-17T09:21:28.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:28 smithi003 bash[25870]: 9b15ead85423fee8d984a43bc8035cfa2a7109fc37a0aabd624aa117ef98a0e4 2024-09-17T09:21:28.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:28 smithi003 systemd[1]: Started Ceph mon.a for 08f126da-74d6-11ef-bceb-c7b262605968. 2024-09-17T09:21:28.032 INFO:teuthology.orchestra.run.smithi003.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-17T09:21:28.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:28 smithi003 ceph-mon[25908]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-17T09:21:28.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:28 smithi003 ceph-mon[25908]: monmap e1: 1 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T09:21:28.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:28 smithi003 ceph-mon[25908]: fsmap 2024-09-17T09:21:28.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:28 smithi003 ceph-mon[25908]: osdmap e1: 0 total, 0 up, 0 in 2024-09-17T09:21:28.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:28 smithi003 ceph-mon[25908]: mgrmap e1: no daemons active 2024-09-17T09:21:29.040 INFO:teuthology.orchestra.run.smithi003.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-17T09:21:29.061 INFO:teuthology.orchestra.run.smithi003.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-17T09:21:29.061 INFO:teuthology.orchestra.run.smithi003.stdout:Creating mgr... 2024-09-17T09:21:29.062 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-17T09:21:29.062 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-17T09:21:29.327 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-08f126da-74d6-11ef-bceb-c7b262605968@mgr.y 2024-09-17T09:21:29.327 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to reset failed state of unit ceph-08f126da-74d6-11ef-bceb-c7b262605968@mgr.y.service: Unit ceph-08f126da-74d6-11ef-bceb-c7b262605968@mgr.y.service not loaded. 2024-09-17T09:21:29.484 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-08f126da-74d6-11ef-bceb-c7b262605968.target.wants/ceph-08f126da-74d6-11ef-bceb-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-08f126da-74d6-11ef-bceb-c7b262605968@.service. 2024-09-17T09:21:29.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:29 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/1627548747' entity='client.admin' 2024-09-17T09:21:29.758 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:29 smithi003 podman[26144]: 2024-09-17 09:21:29.721198568 +0000 UTC m=+0.119361309 container create e24d87b96bc5ef7e85a104982e7819416c3429a915370b40c4b8923e277b8e8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-17T09:21:30.055 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T09:21:30.056 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-09-17T09:21:30.063 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T09:21:30.064 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-09-17T09:21:30.064 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-09-17T09:21:30.064 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to enable service . firewalld.service is not available 2024-09-17T09:21:30.072 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T09:21:30.072 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-09-17T09:21:30.080 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T09:21:30.081 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-09-17T09:21:30.081 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-09-17T09:21:30.081 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-17T09:21:30.081 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr to start... 2024-09-17T09:21:30.081 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr... 2024-09-17T09:21:30.205 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:29 smithi003 podman[26144]: 2024-09-17 09:21:29.940218193 +0000 UTC m=+0.338380943 container init e24d87b96bc5ef7e85a104982e7819416c3429a915370b40c4b8923e277b8e8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-09-17T09:21:30.205 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:29 smithi003 podman[26144]: 2024-09-17 09:21:29.94553021 +0000 UTC m=+0.343692951 container start e24d87b96bc5ef7e85a104982e7819416c3429a915370b40c4b8923e277b8e8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b) 2024-09-17T09:21:30.205 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:30 smithi003 bash[26144]: e24d87b96bc5ef7e85a104982e7819416c3429a915370b40c4b8923e277b8e8f 2024-09-17T09:21:30.205 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:30 smithi003 systemd[1]: Started Ceph mgr.y for 08f126da-74d6-11ef-bceb-c7b262605968. 2024-09-17T09:21:30.205 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:30 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:30.071+0000 7fb778030040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T09:21:30.205 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:30 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:30.133+0000 7fb778030040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T09:21:30.457 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:30 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:30.292+0000 7fb778030040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T09:21:30.457 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:30 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:30.364+0000 7fb778030040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T09:21:30.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:30 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/3194738303' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-17T09:21:30.758 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:30 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:30.518+0000 7fb778030040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:21:30.758 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:30 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:30.669+0000 7fb778030040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T09:21:31.247 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:31 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:31.000+0000 7fb778030040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T09:21:31.247 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:31 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:31.062+0000 7fb778030040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T09:21:31.253 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-17T09:21:31.253 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-09-17T09:21:31.253 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsid": "08f126da-74d6-11ef-bceb-c7b262605968", 2024-09-17T09:21:31.253 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "health": { 2024-09-17T09:21:31.253 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-17T09:21:31.253 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-17T09:21:31.253 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-17T09:21:31.254 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:31.254 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-17T09:21:31.254 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-17T09:21:31.254 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 0 2024-09-17T09:21:31.254 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-17T09:21:31.254 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-17T09:21:31.254 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "a" 2024-09-17T09:21:31.254 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-17T09:21:31.254 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-17T09:21:31.254 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-17T09:21:31.254 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:31.254 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-17T09:21:31.254 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-17T09:21:31.254 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:31.254 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-17T09:21:31.255 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:31.255 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-17T09:21:31.255 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-17T09:21:31.255 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-17T09:21:31.255 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-17T09:21:31.255 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-17T09:21:31.255 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-17T09:21:31.255 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:31.255 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-17T09:21:31.255 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-17T09:21:31.255 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-17T09:21:31.255 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-17T09:21:31.255 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-17T09:21:31.255 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-17T09:21:31.256 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-17T09:21:31.256 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-17T09:21:31.256 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-17T09:21:31.256 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:31.256 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-17T09:21:31.256 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:31.256 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-17T09:21:31.256 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-17T09:21:31.256 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:31.258 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-17T09:21:31.258 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-17T09:21:31.258 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-17T09:21:31.258 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-17T09:21:31.258 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-17T09:21:31.258 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-17T09:21:31.258 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "restful" 2024-09-17T09:21:31.258 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-17T09:21:31.259 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T09:21:31.259 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:31.259 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-17T09:21:31.259 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:31.259 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modified": "2024-09-17T09:21:22.992596+0000", 2024-09-17T09:21:31.259 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T09:21:31.259 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:31.259 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-17T09:21:31.259 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-09-17T09:21:31.259 INFO:teuthology.orchestra.run.smithi003.stdout:mgr not available, waiting (1/15)... 2024-09-17T09:21:31.508 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:31 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:31.254+0000 7fb778030040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T09:21:32.508 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:32 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:32.163+0000 7fb778030040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T09:21:32.508 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:32 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:32.231+0000 7fb778030040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T09:21:33.253 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:32 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:32.985+0000 7fb778030040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T09:21:33.253 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:33 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:33.105+0000 7fb778030040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T09:21:33.253 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:33 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:33.252+0000 7fb778030040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:21:33.508 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:33 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:33.318+0000 7fb778030040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T09:21:33.508 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:33 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:33.480+0000 7fb778030040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T09:21:33.816 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:33 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:33.552+0000 7fb778030040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T09:21:33.816 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:33 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:33.614+0000 7fb778030040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T09:21:33.816 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:33 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:33.782+0000 7fb778030040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T09:21:34.072 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:33 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/2455772497' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-17T09:21:34.073 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:33 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:33.853+0000 7fb778030040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T09:21:34.073 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:33 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:33.922+0000 7fb778030040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T09:21:34.073 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:34 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:34.010+0000 7fb778030040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T09:21:34.073 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:34 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:34.071+0000 7fb778030040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T09:21:34.328 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:34 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:34.134+0000 7fb778030040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T09:21:34.335 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-17T09:21:34.335 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-09-17T09:21:34.335 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsid": "08f126da-74d6-11ef-bceb-c7b262605968", 2024-09-17T09:21:34.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "health": { 2024-09-17T09:21:34.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-17T09:21:34.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-17T09:21:34.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-17T09:21:34.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:34.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-17T09:21:34.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-17T09:21:34.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 0 2024-09-17T09:21:34.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-17T09:21:34.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-17T09:21:34.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "a" 2024-09-17T09:21:34.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-17T09:21:34.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-17T09:21:34.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-17T09:21:34.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:34.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-17T09:21:34.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-17T09:21:34.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:34.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-17T09:21:34.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:34.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-17T09:21:34.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-17T09:21:34.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-17T09:21:34.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-17T09:21:34.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-17T09:21:34.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-17T09:21:34.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:34.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-17T09:21:34.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-17T09:21:34.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-17T09:21:34.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-17T09:21:34.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-17T09:21:34.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-17T09:21:34.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-17T09:21:34.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-17T09:21:34.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-17T09:21:34.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:34.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-17T09:21:34.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:34.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-17T09:21:34.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-17T09:21:34.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:34.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-17T09:21:34.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-17T09:21:34.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-17T09:21:34.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-17T09:21:34.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-17T09:21:34.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-17T09:21:34.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "restful" 2024-09-17T09:21:34.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-17T09:21:34.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T09:21:34.340 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:34.340 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-17T09:21:34.340 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:34.340 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modified": "2024-09-17T09:21:22.992596+0000", 2024-09-17T09:21:34.340 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T09:21:34.340 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:34.340 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-17T09:21:34.340 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-09-17T09:21:34.340 INFO:teuthology.orchestra.run.smithi003.stdout:mgr not available, waiting (2/15)... 2024-09-17T09:21:35.258 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:34 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:34.949+0000 7fb778030040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T09:21:35.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:35 smithi003 ceph-mon[25908]: Activating manager daemon y 2024-09-17T09:21:35.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:35 smithi003 ceph-mon[25908]: mgrmap e2: y(active, starting, since 0.00294423s) 2024-09-17T09:21:35.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:35 smithi003 ceph-mon[25908]: from='mgr.14100 172.21.15.3:0/4083409507' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T09:21:35.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:35 smithi003 ceph-mon[25908]: from='mgr.14100 172.21.15.3:0/4083409507' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T09:21:35.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:35 smithi003 ceph-mon[25908]: from='mgr.14100 172.21.15.3:0/4083409507' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T09:21:35.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:35 smithi003 ceph-mon[25908]: from='mgr.14100 172.21.15.3:0/4083409507' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:21:35.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:35 smithi003 ceph-mon[25908]: from='mgr.14100 172.21.15.3:0/4083409507' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T09:21:35.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:35 smithi003 ceph-mon[25908]: Manager daemon y is now available 2024-09-17T09:21:35.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:35 smithi003 ceph-mon[25908]: from='mgr.14100 172.21.15.3:0/4083409507' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T09:21:35.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:35 smithi003 ceph-mon[25908]: from='mgr.14100 172.21.15.3:0/4083409507' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T09:21:35.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:35 smithi003 ceph-mon[25908]: from='mgr.14100 172.21.15.3:0/4083409507' entity='mgr.y' 2024-09-17T09:21:35.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:35 smithi003 ceph-mon[25908]: from='mgr.14100 172.21.15.3:0/4083409507' entity='mgr.y' 2024-09-17T09:21:35.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:35 smithi003 ceph-mon[25908]: from='mgr.14100 172.21.15.3:0/4083409507' entity='mgr.y' 2024-09-17T09:21:36.964 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:36 smithi003 ceph-mon[25908]: mgrmap e3: y(active, since 1.00964s) 2024-09-17T09:21:37.689 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-17T09:21:37.689 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-09-17T09:21:37.689 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsid": "08f126da-74d6-11ef-bceb-c7b262605968", 2024-09-17T09:21:37.689 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "health": { 2024-09-17T09:21:37.689 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-17T09:21:37.689 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-17T09:21:37.689 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-17T09:21:37.689 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:37.689 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-17T09:21:37.689 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-17T09:21:37.689 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 0 2024-09-17T09:21:37.689 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-17T09:21:37.689 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-17T09:21:37.690 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "a" 2024-09-17T09:21:37.690 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-17T09:21:37.690 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-17T09:21:37.690 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-17T09:21:37.690 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:37.690 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-17T09:21:37.690 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-17T09:21:37.690 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:37.690 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-17T09:21:37.690 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:37.690 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-17T09:21:37.690 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-17T09:21:37.690 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-17T09:21:37.690 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-17T09:21:37.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-17T09:21:37.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-17T09:21:37.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:37.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-17T09:21:37.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-17T09:21:37.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-17T09:21:37.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-17T09:21:37.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-17T09:21:37.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-17T09:21:37.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-17T09:21:37.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-17T09:21:37.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-17T09:21:37.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:37.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-17T09:21:37.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:37.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-17T09:21:37.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-17T09:21:37.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:37.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-17T09:21:37.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-17T09:21:37.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-17T09:21:37.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-17T09:21:37.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-17T09:21:37.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-17T09:21:37.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "restful" 2024-09-17T09:21:37.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-17T09:21:37.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T09:21:37.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:37.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-17T09:21:37.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:37.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modified": "2024-09-17T09:21:22.992596+0000", 2024-09-17T09:21:37.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T09:21:37.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:37.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-17T09:21:37.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-09-17T09:21:37.696 INFO:teuthology.orchestra.run.smithi003.stdout:mgr is available 2024-09-17T09:21:38.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:37 smithi003 ceph-mon[25908]: mgrmap e4: y(active, since 2s) 2024-09-17T09:21:38.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:37 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/711173802' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-17T09:21:38.919 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-17T09:21:38.919 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [global] 2024-09-17T09:21:38.919 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout fsid = 08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:21:38.919 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-17T09:21:38.919 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-17T09:21:38.919 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-17T09:21:38.919 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-17T09:21:38.919 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-17T09:21:38.919 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-17T09:21:38.919 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-17T09:21:38.919 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-17T09:21:38.919 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [osd] 2024-09-17T09:21:38.920 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-17T09:21:38.920 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-17T09:21:38.920 INFO:teuthology.orchestra.run.smithi003.stdout:Enabling cephadm module... 2024-09-17T09:21:39.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:38 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/294355361' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-17T09:21:39.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:38 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/294355361' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-17T09:21:40.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:39 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/729786832' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-17T09:21:40.258 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:40 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: ignoring --setuser ceph since I am not root 2024-09-17T09:21:40.258 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:40 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: ignoring --setgroup ceph since I am not root 2024-09-17T09:21:40.258 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:40 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:40.114+0000 7f8ab4801040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T09:21:40.259 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:40 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:40.172+0000 7f8ab4801040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T09:21:40.547 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:40 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:40.326+0000 7f8ab4801040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T09:21:40.547 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:40 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:40.394+0000 7f8ab4801040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T09:21:40.547 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:40 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:40.545+0000 7f8ab4801040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:21:40.804 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:40 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:40.695+0000 7f8ab4801040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T09:21:41.245 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:40 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/729786832' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-17T09:21:41.245 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:40 smithi003 ceph-mon[25908]: mgrmap e5: y(active, since 5s) 2024-09-17T09:21:41.246 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:41 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:41.032+0000 7f8ab4801040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T09:21:41.246 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:41 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:41.099+0000 7f8ab4801040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T09:21:41.508 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:41 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:41.292+0000 7f8ab4801040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T09:21:41.830 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-09-17T09:21:41.830 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-17T09:21:41.830 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-17T09:21:41.830 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-17T09:21:41.830 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-17T09:21:41.830 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-09-17T09:21:41.830 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for the mgr to restart... 2024-09-17T09:21:41.831 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr epoch 5... 2024-09-17T09:21:42.076 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:41 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/4118264188' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-17T09:21:42.508 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:42 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:42.199+0000 7f8ab4801040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T09:21:42.508 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:42 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:42.269+0000 7f8ab4801040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T09:21:43.258 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:43 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:43.003+0000 7f8ab4801040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T09:21:43.258 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:43 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:43.122+0000 7f8ab4801040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T09:21:43.553 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:43 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:43.268+0000 7f8ab4801040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:21:43.554 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:43 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:43.330+0000 7f8ab4801040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T09:21:43.554 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:43 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:43.482+0000 7f8ab4801040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T09:21:43.554 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:43 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:43.551+0000 7f8ab4801040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T09:21:43.843 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:43 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:43.613+0000 7f8ab4801040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T09:21:43.843 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:43 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:43.774+0000 7f8ab4801040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T09:21:43.843 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:43 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:43.841+0000 7f8ab4801040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T09:21:44.117 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:43 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:43.909+0000 7f8ab4801040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T09:21:44.117 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:43 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:43.994+0000 7f8ab4801040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T09:21:44.117 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:44 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:44.055+0000 7f8ab4801040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T09:21:44.117 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:44 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:44.115+0000 7f8ab4801040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T09:21:45.258 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:44 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:44.937+0000 7f8ab4801040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T09:21:45.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:44 smithi003 ceph-mon[25908]: Active manager daemon y restarted 2024-09-17T09:21:45.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:44 smithi003 ceph-mon[25908]: Activating manager daemon y 2024-09-17T09:21:45.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:44 smithi003 ceph-mon[25908]: osdmap e2: 0 total, 0 up, 0 in 2024-09-17T09:21:45.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:44 smithi003 ceph-mon[25908]: mgrmap e6: y(active, starting, since 0.00348683s) 2024-09-17T09:21:45.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:44 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:21:45.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:44 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T09:21:45.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:44 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T09:21:45.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:44 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T09:21:45.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:44 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T09:21:45.260 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:44 smithi003 ceph-mon[25908]: Manager daemon y is now available 2024-09-17T09:21:45.260 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:44 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' 2024-09-17T09:21:45.260 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:44 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' 2024-09-17T09:21:45.260 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:44 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:21:45.260 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:44 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:21:45.260 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:44 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T09:21:46.257 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:45 smithi003 ceph-mon[25908]: Found migration_current of "None". Setting to last migration. 2024-09-17T09:21:46.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:45 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T09:21:46.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:45 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' 2024-09-17T09:21:46.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:45 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' 2024-09-17T09:21:46.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:45 smithi003 ceph-mon[25908]: mgrmap e7: y(active, since 1.00546s) 2024-09-17T09:21:46.543 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-09-17T09:21:46.543 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-17T09:21:46.543 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-17T09:21:46.543 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-09-17T09:21:46.543 INFO:teuthology.orchestra.run.smithi003.stdout:mgr epoch 5 is available 2024-09-17T09:21:46.543 INFO:teuthology.orchestra.run.smithi003.stdout:Setting orchestrator backend to cephadm... 2024-09-17T09:21:47.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:46 smithi003 ceph-mon[25908]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-17T09:21:47.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:46 smithi003 ceph-mon[25908]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-17T09:21:47.997 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:47.752+0000 7f8a1c4e1640 -1 log_channel(cephadm) log [ERR] : [17/Sep/2024:09:21:47] ENGINE Error in HTTPServer.serve 2024-09-17T09:21:47.997 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: Traceback (most recent call last): 2024-09-17T09:21:47.997 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T09:21:47.997 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: self._connections.run(self.expiration_interval) 2024-09-17T09:21:47.998 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T09:21:47.998 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: self._run(expiration_interval) 2024-09-17T09:21:47.998 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T09:21:47.998 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T09:21:47.998 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T09:21:47.998 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T09:21:47.998 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T09:21:47.998 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: s = self.context.wrap_socket( 2024-09-17T09:21:47.998 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T09:21:47.998 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: return self.sslsocket_class._create( 2024-09-17T09:21:47.998 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T09:21:47.998 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: self.do_handshake() 2024-09-17T09:21:47.999 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T09:21:47.999 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: self._sslobj.do_handshake() 2024-09-17T09:21:47.999 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T09:21:47.999 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:21:48.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-mon[25908]: mgrmap e8: y(active, since 2s) 2024-09-17T09:21:48.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' 2024-09-17T09:21:48.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:21:48.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:47 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:21:48.996 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-17T09:21:48.996 INFO:teuthology.orchestra.run.smithi003.stdout:Generating ssh key... 2024-09-17T09:21:49.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:48 smithi003 ceph-mon[25908]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:21:49.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:48 smithi003 ceph-mon[25908]: [17/Sep/2024:09:21:47] ENGINE Bus STARTING 2024-09-17T09:21:49.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:48 smithi003 ceph-mon[25908]: [17/Sep/2024:09:21:47] ENGINE Serving on http://172.21.15.3:8765 2024-09-17T09:21:49.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:48 smithi003 ceph-mon[25908]: [17/Sep/2024:09:21:47] ENGINE Serving on https://172.21.15.3:7150 2024-09-17T09:21:49.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:48 smithi003 ceph-mon[25908]: [17/Sep/2024:09:21:47] ENGINE Bus STARTED 2024-09-17T09:21:49.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:48 smithi003 ceph-mon[25908]: [17/Sep/2024:09:21:47] ENGINE Error in HTTPServer.serve 2024-09-17T09:21:49.253 INFO:journalctl@ceph.mon.a.smithi003.stdout: Traceback (most recent call last): 2024-09-17T09:21:49.253 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T09:21:49.253 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._connections.run(self.expiration_interval) 2024-09-17T09:21:49.253 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T09:21:49.253 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._run(expiration_interval) 2024-09-17T09:21:49.253 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T09:21:49.253 INFO:journalctl@ceph.mon.a.smithi003.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T09:21:49.253 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T09:21:49.253 INFO:journalctl@ceph.mon.a.smithi003.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T09:21:49.253 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T09:21:49.253 INFO:journalctl@ceph.mon.a.smithi003.stdout: s = self.context.wrap_socket( 2024-09-17T09:21:49.253 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T09:21:49.253 INFO:journalctl@ceph.mon.a.smithi003.stdout: return self.sslsocket_class._create( 2024-09-17T09:21:49.253 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T09:21:49.254 INFO:journalctl@ceph.mon.a.smithi003.stdout: self.do_handshake() 2024-09-17T09:21:49.254 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T09:21:49.254 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._sslobj.do_handshake() 2024-09-17T09:21:49.254 INFO:journalctl@ceph.mon.a.smithi003.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T09:21:50.001 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:49 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: Generating public/private rsa key pair. 2024-09-17T09:21:50.001 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:49 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: Your identification has been saved in /tmp/tmpiafbahiy/key 2024-09-17T09:21:50.001 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:49 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: Your public key has been saved in /tmp/tmpiafbahiy/key.pub 2024-09-17T09:21:50.001 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:49 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: The key fingerprint is: 2024-09-17T09:21:50.001 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:49 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: SHA256:kPwo3DDHAmRM4XU3ESs//9WG2uZ3VGinUp1fNYXzjaU ceph-08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:21:50.001 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:49 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: The key's randomart image is: 2024-09-17T09:21:50.001 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:49 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: +---[RSA 3072]----+ 2024-09-17T09:21:50.001 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:49 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: | +*.. . =o .o| 2024-09-17T09:21:50.001 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:49 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: | o.o + o o o.o| 2024-09-17T09:21:50.001 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:49 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: | . + B . O=| 2024-09-17T09:21:50.001 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:49 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: | . * * E.B| 2024-09-17T09:21:50.002 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:49 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: | o o S o o+| 2024-09-17T09:21:50.002 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:49 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: | . o . .oo| 2024-09-17T09:21:50.002 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:49 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: | . .o.o| 2024-09-17T09:21:50.002 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:49 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: | . +..o| 2024-09-17T09:21:50.002 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:49 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: | ooo..| 2024-09-17T09:21:50.002 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:21:49 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: +----[SHA256]-----+ 2024-09-17T09:21:50.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:50 smithi003 ceph-mon[25908]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:21:50.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:50 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' 2024-09-17T09:21:50.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:50 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' 2024-09-17T09:21:51.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:51 smithi003 ceph-mon[25908]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:21:51.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:51 smithi003 ceph-mon[25908]: Generating ssh key... 2024-09-17T09:21:51.527 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuACT09AnnF+onyzLpGobT2QN0WM1FIO6RWKA70PSkIoKyUL527ojJtBK/dzAefil3QfdJmVvD7UXXqrwYfCOa09xmEBnh6CWnF/IrrpmyOia+QNpkTi97HHmim/dIjgkbkgF1AFKHMN3DX3fOOJr6BF4AeBLGmLp/juHOnbivZTQOt5IRyDTXm5qVFFRhOIhaKh4lus31+BFVIgf0pL6Tbu5Jca+2rvhGVqO85qQZMXNtjYcrnpcyk5IMgtAb6CawVxoRjRaIGMxmbdFn8zH+bgp72tUnb8l4KqnAL4ZCeau6aIpMJZYur2mFKU5oVLlfVZYgs5E1aaeGK+DA7EUVf+frDPQfe9nKJLSw7KAPZUUOpZ2jVRmoxnHhhtWfKBHdo36slWL5cZGFZiDV+9wkyEz7ahKicGyWg8qov5Bm/CM/Gmj1w6NVu25HVpFCbqqokIcILIJns1GP9jZLbPz1KhMSs0K756NnOR5O3gWSzYIHV99dXtPFUISpGD4sFOs= ceph-08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:21:51.527 INFO:teuthology.orchestra.run.smithi003.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-17T09:21:51.527 INFO:teuthology.orchestra.run.smithi003.stdout:Adding key to root@localhost authorized_keys... 2024-09-17T09:21:51.533 INFO:teuthology.orchestra.run.smithi003.stdout:Adding host smithi003... 2024-09-17T09:21:52.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:52 smithi003 ceph-mon[25908]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:21:53.213 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:53 smithi003 ceph-mon[25908]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi003", "addr": "172.21.15.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:21:54.156 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:54 smithi003 ceph-mon[25908]: Deploying cephadm binary to smithi003 2024-09-17T09:21:56.125 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Added host 'smithi003' with addr '172.21.15.3' 2024-09-17T09:21:56.126 INFO:teuthology.orchestra.run.smithi003.stdout:Deploying unmanaged mon service... 2024-09-17T09:21:56.686 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:56 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' 2024-09-17T09:21:56.686 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:56 smithi003 ceph-mon[25908]: Added host smithi003 2024-09-17T09:21:56.686 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:56 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:21:57.761 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-17T09:21:57.761 INFO:teuthology.orchestra.run.smithi003.stdout:Deploying unmanaged mgr service... 2024-09-17T09:21:58.021 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:57 smithi003 ceph-mon[25908]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:21:58.021 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:57 smithi003 ceph-mon[25908]: Saving service mon spec with placement count:5 2024-09-17T09:21:58.021 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:57 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' 2024-09-17T09:21:59.369 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-17T09:21:59.701 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:59 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' 2024-09-17T09:21:59.702 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:59 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' 2024-09-17T09:21:59.702 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:59 smithi003 ceph-mon[25908]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:21:59.702 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:59 smithi003 ceph-mon[25908]: Saving service mgr spec with placement count:2 2024-09-17T09:21:59.702 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:21:59 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' 2024-09-17T09:22:01.204 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:00 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/3146671058' entity='client.admin' 2024-09-17T09:22:01.756 INFO:teuthology.orchestra.run.smithi003.stdout:Enabling the dashboard module... 2024-09-17T09:22:02.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:02 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/3195397369' entity='client.admin' 2024-09-17T09:22:03.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:03 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/3941635552' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-17T09:22:03.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:03 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' 2024-09-17T09:22:03.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:03 smithi003 ceph-mon[25908]: from='mgr.14118 172.21.15.3:0/945759117' entity='mgr.y' 2024-09-17T09:22:03.509 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:03 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: ignoring --setuser ceph since I am not root 2024-09-17T09:22:03.509 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:03 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: ignoring --setgroup ceph since I am not root 2024-09-17T09:22:03.509 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:03 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:03.354+0000 7fb616962040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T09:22:03.509 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:03 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:03.413+0000 7fb616962040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T09:22:03.824 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:03 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:03.570+0000 7fb616962040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T09:22:03.824 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:03 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:03.639+0000 7fb616962040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T09:22:03.825 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:03 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:03.790+0000 7fb616962040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:22:04.092 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:03 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:03.941+0000 7fb616962040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T09:22:04.400 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:04 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/3941635552' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-17T09:22:04.400 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:04 smithi003 ceph-mon[25908]: mgrmap e9: y(active, since 18s) 2024-09-17T09:22:04.401 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:04 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:04.272+0000 7fb616962040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T09:22:04.401 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:04 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:04.337+0000 7fb616962040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T09:22:04.758 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:04 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:04.531+0000 7fb616962040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T09:22:05.451 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:05 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/2288870560' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-17T09:22:05.568 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-09-17T09:22:05.568 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-17T09:22:05.568 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-17T09:22:05.568 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-17T09:22:05.568 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-17T09:22:05.569 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-09-17T09:22:05.569 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for the mgr to restart... 2024-09-17T09:22:05.569 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr epoch 9... 2024-09-17T09:22:05.758 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:05 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:05.449+0000 7fb616962040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T09:22:05.791 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:05 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:05.517+0000 7fb616962040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T09:22:06.486 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:06 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:06.277+0000 7fb616962040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T09:22:06.487 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:06 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:06.397+0000 7fb616962040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T09:22:06.758 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:06 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:06.547+0000 7fb616962040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:22:06.758 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:06 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:06.610+0000 7fb616962040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T09:22:07.058 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:06 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:06.762+0000 7fb616962040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T09:22:07.059 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:06 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:06.832+0000 7fb616962040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T09:22:07.059 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:06 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:06.894+0000 7fb616962040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T09:22:07.059 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:07 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:07.056+0000 7fb616962040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T09:22:07.341 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:07 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:07.124+0000 7fb616962040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T09:22:07.341 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:07 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:07.192+0000 7fb616962040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T09:22:07.341 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:07 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:07.278+0000 7fb616962040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T09:22:07.341 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:07 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:07.339+0000 7fb616962040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T09:22:07.758 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:07 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:07.399+0000 7fb616962040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T09:22:08.466 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:08 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:08.209+0000 7fb616962040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T09:22:08.466 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:08 smithi003 ceph-mon[25908]: Active manager daemon y restarted 2024-09-17T09:22:08.467 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:08 smithi003 ceph-mon[25908]: Activating manager daemon y 2024-09-17T09:22:08.467 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:08 smithi003 ceph-mon[25908]: osdmap e3: 0 total, 0 up, 0 in 2024-09-17T09:22:08.467 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:08 smithi003 ceph-mon[25908]: mgrmap e10: y(active, starting, since 0.00307361s) 2024-09-17T09:22:08.467 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:08 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:22:08.467 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:08 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T09:22:08.467 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:08 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T09:22:08.467 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:08 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T09:22:08.467 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:08 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T09:22:08.467 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:08 smithi003 ceph-mon[25908]: Manager daemon y is now available 2024-09-17T09:22:08.467 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:08 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:08.467 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:08 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T09:22:08.467 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:08 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T09:22:10.055 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-09-17T09:22:10.055 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-17T09:22:10.055 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-17T09:22:10.055 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-09-17T09:22:10.055 INFO:teuthology.orchestra.run.smithi003.stdout:mgr epoch 9 is available 2024-09-17T09:22:10.055 INFO:teuthology.orchestra.run.smithi003.stdout:Generating a dashboard self-signed certificate... 2024-09-17T09:22:10.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:10 smithi003 ceph-mon[25908]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-17T09:22:10.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:10 smithi003 ceph-mon[25908]: mgrmap e11: y(active, since 1.00584s) 2024-09-17T09:22:10.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:10 smithi003 ceph-mon[25908]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-17T09:22:10.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:10 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:11.375 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:11.257+0000 7fb57cc2c640 -1 log_channel(cephadm) log [ERR] : [17/Sep/2024:09:22:11] ENGINE Error in HTTPServer.serve 2024-09-17T09:22:11.375 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: Traceback (most recent call last): 2024-09-17T09:22:11.376 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T09:22:11.376 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: self._connections.run(self.expiration_interval) 2024-09-17T09:22:11.376 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T09:22:11.376 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: self._run(expiration_interval) 2024-09-17T09:22:11.376 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T09:22:11.376 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T09:22:11.376 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T09:22:11.376 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T09:22:11.376 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T09:22:11.376 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: s = self.context.wrap_socket( 2024-09-17T09:22:11.376 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T09:22:11.377 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: return self.sslsocket_class._create( 2024-09-17T09:22:11.377 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T09:22:11.377 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: self.do_handshake() 2024-09-17T09:22:11.377 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T09:22:11.377 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: self._sslobj.do_handshake() 2024-09-17T09:22:11.377 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T09:22:11.377 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:11.378 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-17T09:22:11.378 INFO:teuthology.orchestra.run.smithi003.stdout:Creating initial admin user... 2024-09-17T09:22:12.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:12.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-mon[25908]: mgrmap e12: y(active, since 2s) 2024-09-17T09:22:12.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:12.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:12.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-mon[25908]: [17/Sep/2024:09:22:11] ENGINE Bus STARTING 2024-09-17T09:22:12.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-mon[25908]: [17/Sep/2024:09:22:11] ENGINE Serving on http://172.21.15.3:8765 2024-09-17T09:22:12.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-mon[25908]: [17/Sep/2024:09:22:11] ENGINE Serving on https://172.21.15.3:7150 2024-09-17T09:22:12.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:11 smithi003 ceph-mon[25908]: [17/Sep/2024:09:22:11] ENGINE Bus STARTED 2024-09-17T09:22:13.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:12 smithi003 ceph-mon[25908]: [17/Sep/2024:09:22:11] ENGINE Error in HTTPServer.serve 2024-09-17T09:22:13.008 INFO:journalctl@ceph.mon.a.smithi003.stdout: Traceback (most recent call last): 2024-09-17T09:22:13.008 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T09:22:13.008 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._connections.run(self.expiration_interval) 2024-09-17T09:22:13.008 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T09:22:13.008 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._run(expiration_interval) 2024-09-17T09:22:13.009 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T09:22:13.009 INFO:journalctl@ceph.mon.a.smithi003.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T09:22:13.009 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T09:22:13.009 INFO:journalctl@ceph.mon.a.smithi003.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T09:22:13.009 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T09:22:13.009 INFO:journalctl@ceph.mon.a.smithi003.stdout: s = self.context.wrap_socket( 2024-09-17T09:22:13.009 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T09:22:13.009 INFO:journalctl@ceph.mon.a.smithi003.stdout: return self.sslsocket_class._create( 2024-09-17T09:22:13.009 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T09:22:13.009 INFO:journalctl@ceph.mon.a.smithi003.stdout: self.do_handshake() 2024-09-17T09:22:13.009 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T09:22:13.009 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._sslobj.do_handshake() 2024-09-17T09:22:13.010 INFO:journalctl@ceph.mon.a.smithi003.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T09:22:13.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:12 smithi003 ceph-mon[25908]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:22:13.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:12 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:13.665 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$F/47L.9HBBDYiJ0nQTLMOOUyuo2ixxOzxRDbY4nYdXt1ZynfBq9ka", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726564932, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-17T09:22:13.666 INFO:teuthology.orchestra.run.smithi003.stdout:Fetching dashboard port number... 2024-09-17T09:22:14.896 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 8443 2024-09-17T09:22:14.907 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T09:22:14.907 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-09-17T09:22:14.915 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T09:22:14.915 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-09-17T09:22:14.915 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-09-17T09:22:14.915 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-17T09:22:14.917 INFO:teuthology.orchestra.run.smithi003.stdout:Ceph Dashboard is now available at: 2024-09-17T09:22:14.917 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:22:14.917 INFO:teuthology.orchestra.run.smithi003.stdout: URL: https://smithi003.front.sepia.ceph.com:8443/ 2024-09-17T09:22:14.917 INFO:teuthology.orchestra.run.smithi003.stdout: User: admin 2024-09-17T09:22:14.917 INFO:teuthology.orchestra.run.smithi003.stdout: Password: m0milwy4i0 2024-09-17T09:22:14.917 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:22:14.918 INFO:teuthology.orchestra.run.smithi003.stdout:Saving cluster configuration to /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config directory 2024-09-17T09:22:14.918 INFO:teuthology.orchestra.run.smithi003.stdout:Enabling autotune for osd_memory_target 2024-09-17T09:22:15.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:15 smithi003 ceph-mon[25908]: mgrmap e13: y(active, since 6s) 2024-09-17T09:22:15.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:15 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/1952840838' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-17T09:22:17.450 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-17T09:22:17.450 INFO:teuthology.orchestra.run.smithi003.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-17T09:22:17.451 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:22:17.451 INFO:teuthology.orchestra.run.smithi003.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-17T09:22:17.451 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:22:17.451 INFO:teuthology.orchestra.run.smithi003.stdout:Or, if you are only running a single cluster on this host: 2024-09-17T09:22:17.451 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:22:17.451 INFO:teuthology.orchestra.run.smithi003.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-17T09:22:17.451 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:22:17.451 INFO:teuthology.orchestra.run.smithi003.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-17T09:22:17.451 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:22:17.451 INFO:teuthology.orchestra.run.smithi003.stdout: ceph telemetry on 2024-09-17T09:22:17.451 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:22:17.451 INFO:teuthology.orchestra.run.smithi003.stdout:For more information see: 2024-09-17T09:22:17.451 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:22:17.452 INFO:teuthology.orchestra.run.smithi003.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-17T09:22:17.452 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:22:17.452 INFO:teuthology.orchestra.run.smithi003.stdout:Bootstrap complete. 2024-09-17T09:22:17.486 INFO:tasks.cephadm:Fetching config... 2024-09-17T09:22:17.486 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-17T09:22:17.486 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-17T09:22:17.503 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-17T09:22:17.503 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-17T09:22:17.503 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-17T09:22:17.560 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-17T09:22:17.560 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-17T09:22:17.560 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-17T09:22:17.626 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-17T09:22:17.626 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-17T09:22:17.626 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-17T09:22:17.683 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-17T09:22:17.683 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuACT09AnnF+onyzLpGobT2QN0WM1FIO6RWKA70PSkIoKyUL527ojJtBK/dzAefil3QfdJmVvD7UXXqrwYfCOa09xmEBnh6CWnF/IrrpmyOia+QNpkTi97HHmim/dIjgkbkgF1AFKHMN3DX3fOOJr6BF4AeBLGmLp/juHOnbivZTQOt5IRyDTXm5qVFFRhOIhaKh4lus31+BFVIgf0pL6Tbu5Jca+2rvhGVqO85qQZMXNtjYcrnpcyk5IMgtAb6CawVxoRjRaIGMxmbdFn8zH+bgp72tUnb8l4KqnAL4ZCeau6aIpMJZYur2mFKU5oVLlfVZYgs5E1aaeGK+DA7EUVf+frDPQfe9nKJLSw7KAPZUUOpZ2jVRmoxnHhhtWfKBHdo36slWL5cZGFZiDV+9wkyEz7ahKicGyWg8qov5Bm/CM/Gmj1w6NVu25HVpFCbqqokIcILIJns1GP9jZLbPz1KhMSs0K756NnOR5O3gWSzYIHV99dXtPFUISpGD4sFOs= ceph-08f126da-74d6-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-17T09:22:17.790 INFO:teuthology.orchestra.run.smithi003.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuACT09AnnF+onyzLpGobT2QN0WM1FIO6RWKA70PSkIoKyUL527ojJtBK/dzAefil3QfdJmVvD7UXXqrwYfCOa09xmEBnh6CWnF/IrrpmyOia+QNpkTi97HHmim/dIjgkbkgF1AFKHMN3DX3fOOJr6BF4AeBLGmLp/juHOnbivZTQOt5IRyDTXm5qVFFRhOIhaKh4lus31+BFVIgf0pL6Tbu5Jca+2rvhGVqO85qQZMXNtjYcrnpcyk5IMgtAb6CawVxoRjRaIGMxmbdFn8zH+bgp72tUnb8l4KqnAL4ZCeau6aIpMJZYur2mFKU5oVLlfVZYgs5E1aaeGK+DA7EUVf+frDPQfe9nKJLSw7KAPZUUOpZ2jVRmoxnHhhtWfKBHdo36slWL5cZGFZiDV+9wkyEz7ahKicGyWg8qov5Bm/CM/Gmj1w6NVu25HVpFCbqqokIcILIJns1GP9jZLbPz1KhMSs0K756NnOR5O3gWSzYIHV99dXtPFUISpGD4sFOs= ceph-08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:22:17.803 DEBUG:teuthology.orchestra.run.smithi077:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuACT09AnnF+onyzLpGobT2QN0WM1FIO6RWKA70PSkIoKyUL527ojJtBK/dzAefil3QfdJmVvD7UXXqrwYfCOa09xmEBnh6CWnF/IrrpmyOia+QNpkTi97HHmim/dIjgkbkgF1AFKHMN3DX3fOOJr6BF4AeBLGmLp/juHOnbivZTQOt5IRyDTXm5qVFFRhOIhaKh4lus31+BFVIgf0pL6Tbu5Jca+2rvhGVqO85qQZMXNtjYcrnpcyk5IMgtAb6CawVxoRjRaIGMxmbdFn8zH+bgp72tUnb8l4KqnAL4ZCeau6aIpMJZYur2mFKU5oVLlfVZYgs5E1aaeGK+DA7EUVf+frDPQfe9nKJLSw7KAPZUUOpZ2jVRmoxnHhhtWfKBHdo36slWL5cZGFZiDV+9wkyEz7ahKicGyWg8qov5Bm/CM/Gmj1w6NVu25HVpFCbqqokIcILIJns1GP9jZLbPz1KhMSs0K756NnOR5O3gWSzYIHV99dXtPFUISpGD4sFOs= ceph-08f126da-74d6-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-17T09:22:17.919 INFO:teuthology.orchestra.run.smithi077.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuACT09AnnF+onyzLpGobT2QN0WM1FIO6RWKA70PSkIoKyUL527ojJtBK/dzAefil3QfdJmVvD7UXXqrwYfCOa09xmEBnh6CWnF/IrrpmyOia+QNpkTi97HHmim/dIjgkbkgF1AFKHMN3DX3fOOJr6BF4AeBLGmLp/juHOnbivZTQOt5IRyDTXm5qVFFRhOIhaKh4lus31+BFVIgf0pL6Tbu5Jca+2rvhGVqO85qQZMXNtjYcrnpcyk5IMgtAb6CawVxoRjRaIGMxmbdFn8zH+bgp72tUnb8l4KqnAL4ZCeau6aIpMJZYur2mFKU5oVLlfVZYgs5E1aaeGK+DA7EUVf+frDPQfe9nKJLSw7KAPZUUOpZ2jVRmoxnHhhtWfKBHdo36slWL5cZGFZiDV+9wkyEz7ahKicGyWg8qov5Bm/CM/Gmj1w6NVu25HVpFCbqqokIcILIJns1GP9jZLbPz1KhMSs0K756NnOR5O3gWSzYIHV99dXtPFUISpGD4sFOs= ceph-08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:22:17.932 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuACT09AnnF+onyzLpGobT2QN0WM1FIO6RWKA70PSkIoKyUL527ojJtBK/dzAefil3QfdJmVvD7UXXqrwYfCOa09xmEBnh6CWnF/IrrpmyOia+QNpkTi97HHmim/dIjgkbkgF1AFKHMN3DX3fOOJr6BF4AeBLGmLp/juHOnbivZTQOt5IRyDTXm5qVFFRhOIhaKh4lus31+BFVIgf0pL6Tbu5Jca+2rvhGVqO85qQZMXNtjYcrnpcyk5IMgtAb6CawVxoRjRaIGMxmbdFn8zH+bgp72tUnb8l4KqnAL4ZCeau6aIpMJZYur2mFKU5oVLlfVZYgs5E1aaeGK+DA7EUVf+frDPQfe9nKJLSw7KAPZUUOpZ2jVRmoxnHhhtWfKBHdo36slWL5cZGFZiDV+9wkyEz7ahKicGyWg8qov5Bm/CM/Gmj1w6NVu25HVpFCbqqokIcILIJns1GP9jZLbPz1KhMSs0K756NnOR5O3gWSzYIHV99dXtPFUISpGD4sFOs= ceph-08f126da-74d6-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-17T09:22:17.983 INFO:teuthology.orchestra.run.smithi183.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuACT09AnnF+onyzLpGobT2QN0WM1FIO6RWKA70PSkIoKyUL527ojJtBK/dzAefil3QfdJmVvD7UXXqrwYfCOa09xmEBnh6CWnF/IrrpmyOia+QNpkTi97HHmim/dIjgkbkgF1AFKHMN3DX3fOOJr6BF4AeBLGmLp/juHOnbivZTQOt5IRyDTXm5qVFFRhOIhaKh4lus31+BFVIgf0pL6Tbu5Jca+2rvhGVqO85qQZMXNtjYcrnpcyk5IMgtAb6CawVxoRjRaIGMxmbdFn8zH+bgp72tUnb8l4KqnAL4ZCeau6aIpMJZYur2mFKU5oVLlfVZYgs5E1aaeGK+DA7EUVf+frDPQfe9nKJLSw7KAPZUUOpZ2jVRmoxnHhhtWfKBHdo36slWL5cZGFZiDV+9wkyEz7ahKicGyWg8qov5Bm/CM/Gmj1w6NVu25HVpFCbqqokIcILIJns1GP9jZLbPz1KhMSs0K756NnOR5O3gWSzYIHV99dXtPFUISpGD4sFOs= ceph-08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:22:17.996 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-17T09:22:18.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:17 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/3281364189' entity='client.admin' 2024-09-17T09:22:18.194 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:22:22.044 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-17T09:22:22.044 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-17T09:22:22.511 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:22:22.632 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:22 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/3853495436' entity='client.admin' 2024-09-17T09:22:23.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:23 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:23.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:23 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:23.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:23 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi003", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:22:23.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:23 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:23.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:23 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:23.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:23 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:25.146 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi077 2024-09-17T09:22:25.146 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-17T09:22:25.147 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/etc/ceph/ceph.conf 2024-09-17T09:22:25.167 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-17T09:22:25.167 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:22:25.225 INFO:tasks.cephadm:Adding host smithi077 to orchestrator... 2024-09-17T09:22:25.226 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph orch host add smithi077 2024-09-17T09:22:25.427 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:22:25.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:25 smithi003 ceph-mon[25908]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:22:25.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:25 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:25.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:25 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:25.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:25 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:25.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:25 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:25.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:25 smithi003 ceph-mon[25908]: Updating smithi003:/etc/ceph/ceph.conf 2024-09-17T09:22:25.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:25 smithi003 ceph-mon[25908]: Updating smithi003:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:22:25.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:25 smithi003 ceph-mon[25908]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:22:25.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:25 smithi003 ceph-mon[25908]: Updating smithi003:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:22:25.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:25 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:25.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:25 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:25.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:25 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:27.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:27 smithi003 ceph-mon[25908]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi077", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:22:29.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:29 smithi003 ceph-mon[25908]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:30.452 INFO:teuthology.orchestra.run.smithi003.stdout:Added host 'smithi077' with addr '172.21.15.77' 2024-09-17T09:22:30.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:30 smithi003 ceph-mon[25908]: Deploying cephadm binary to smithi077 2024-09-17T09:22:31.009 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-17T09:22:31.204 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:22:31.325 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:31 smithi003 ceph-mon[25908]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:31.325 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:31 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:31.325 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:31 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:31.325 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:31 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:31.325 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:31 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:32.317 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:32 smithi003 ceph-mon[25908]: Added host smithi077 2024-09-17T09:22:32.317 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:32 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:33.031 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:22:33.031 INFO:teuthology.orchestra.run.smithi003.stdout:[{"addr": "172.21.15.3", "hostname": "smithi003", "labels": [], "status": ""}, {"addr": "172.21.15.77", "hostname": "smithi077", "labels": [], "status": ""}] 2024-09-17T09:22:33.357 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:33 smithi003 ceph-mon[25908]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:33.656 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi183 2024-09-17T09:22:33.656 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-17T09:22:33.656 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.conf 2024-09-17T09:22:33.675 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-17T09:22:33.676 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:22:33.734 INFO:tasks.cephadm:Adding host smithi183 to orchestrator... 2024-09-17T09:22:33.734 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph orch host add smithi183 2024-09-17T09:22:33.936 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:22:34.385 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:34 smithi003 ceph-mon[25908]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T09:22:35.325 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:35 smithi003 ceph-mon[25908]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:35.325 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:35 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:35.325 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:35 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:35.325 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:35 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:35.325 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:35 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:35.325 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:35 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi077", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:22:35.325 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:35 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:35.325 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:35 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:36.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:36 smithi003 ceph-mon[25908]: Updating smithi077:/etc/ceph/ceph.conf 2024-09-17T09:22:36.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:36 smithi003 ceph-mon[25908]: Updating smithi077:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:22:36.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:36 smithi003 ceph-mon[25908]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:22:36.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:36 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:36.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:36 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:36.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:36 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:37.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:37 smithi003 ceph-mon[25908]: Updating smithi077:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:22:37.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:37 smithi003 ceph-mon[25908]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:22:37.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:37 smithi003 ceph-mon[25908]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:38.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:38 smithi003 ceph-mon[25908]: Deploying cephadm binary to smithi183 2024-09-17T09:22:39.043 INFO:teuthology.orchestra.run.smithi003.stdout:Added host 'smithi183' with addr '172.21.15.183' 2024-09-17T09:22:39.353 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:39 smithi003 ceph-mon[25908]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:39.353 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:39 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:39.354 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:39 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:39.667 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-17T09:22:39.859 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:22:40.651 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:40 smithi003 ceph-mon[25908]: Added host smithi183 2024-09-17T09:22:40.651 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:40 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:40.652 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:40 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:41.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:41 smithi003 ceph-mon[25908]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:41.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:41 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:41.636 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:22:41.770 INFO:teuthology.orchestra.run.smithi003.stdout:[{"addr": "172.21.15.3", "hostname": "smithi003", "labels": [], "status": ""}, {"addr": "172.21.15.77", "hostname": "smithi077", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}] 2024-09-17T09:22:42.239 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-17T09:22:42.239 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-17T09:22:42.437 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:22:42.557 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:42 smithi003 ceph-mon[25908]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T09:22:42.558 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:42 smithi003 ceph-mon[25908]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:44.358 INFO:teuthology.orchestra.run.smithi003.stderr:adjusted tunables profile to default 2024-09-17T09:22:44.620 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:44 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:44.621 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:44 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:44.621 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:44 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:44.621 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:44 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:44.621 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:44 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:22:44.621 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:44 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:44.621 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:44 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:44.621 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:44 smithi003 ceph-mon[25908]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-17T09:22:44.621 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:44 smithi003 ceph-mon[25908]: Updating smithi183:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:22:44.621 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:44 smithi003 ceph-mon[25908]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:22:44.621 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:44 smithi003 ceph-mon[25908]: Updating smithi183:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:22:44.622 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:44 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/3376471417' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-17T09:22:44.622 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:44 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:44.622 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:44 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:44.622 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:44 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:44.622 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:44 smithi003 ceph-mon[25908]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:45.048 INFO:tasks.cephadm:Adding mon.a on smithi003 2024-09-17T09:22:45.049 INFO:tasks.cephadm:Adding mon.b on smithi077 2024-09-17T09:22:45.049 INFO:tasks.cephadm:Adding mon.c on smithi183 2024-09-17T09:22:45.050 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi003:172.21.15.3=a;smithi077:172.21.15.77=b;smithi183:172.21.15.183=c' 2024-09-17T09:22:45.234 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:22:45.608 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:45 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/3376471417' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-17T09:22:45.608 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:45 smithi003 ceph-mon[25908]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T09:22:46.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:46 smithi003 ceph-mon[25908]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:46.887 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mon update... 2024-09-17T09:22:47.424 DEBUG:teuthology.orchestra.run.smithi077:mon.b> sudo journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@mon.b.service 2024-09-17T09:22:47.427 DEBUG:teuthology.orchestra.run.smithi183:mon.c> sudo journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@mon.c.service 2024-09-17T09:22:47.431 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-17T09:22:47.432 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-17T09:22:48.151 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:22:48.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:47 smithi003 ceph-mon[25908]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi077:172.21.15.77=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:22:48.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:47 smithi003 ceph-mon[25908]: Saving service mon spec with placement smithi003:172.21.15.3=a;smithi077:172.21.15.77=b;smithi183:172.21.15.183=c;count:3 2024-09-17T09:22:48.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:47 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:48.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:47 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:48.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:47 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:48.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:47 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:48.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:47 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:48.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:47 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:22:48.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:47 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:48.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:47 smithi003 ceph-mon[25908]: Deploying daemon mon.c on smithi183 2024-09-17T09:22:49.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:48 smithi003 ceph-mon[25908]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:50.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:50 smithi003 ceph-mon[25908]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:51.562 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-17T09:22:51.562 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":1,"fsid":"08f126da-74d6-11ef-bceb-c7b262605968","modified":"2024-09-17T09:21:19.203553Z","created":"2024-09-17T09:21:19.203553Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-17T09:22:51.563 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 1 2024-09-17T09:22:51.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:51 smithi183 systemd[1]: Starting Ceph mon.c for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:22:52.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:51 smithi003 ceph-mon[25908]: from='client.? 172.21.15.183:0/2325028841' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T09:22:52.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 podman[30333]: 2024-09-17 09:22:52.372272903 +0000 UTC m=+0.015614678 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T09:22:52.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 podman[30333]: 2024-09-17 09:22:52.471751309 +0000 UTC m=+0.115093130 container create d00a181b864177a30934b3e1367ebaa1945b0573d18ee868b53513979a01fcf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-c, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T09:22:52.725 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 podman[30333]: 2024-09-17 09:22:52.699007222 +0000 UTC m=+0.342349005 container init d00a181b864177a30934b3e1367ebaa1945b0573d18ee868b53513979a01fcf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, RELEASE=reef-19e2a2b, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T09:22:52.725 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 podman[30333]: 2024-09-17 09:22:52.703485776 +0000 UTC m=+0.346827558 container start d00a181b864177a30934b3e1367ebaa1945b0573d18ee868b53513979a01fcf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-17T09:22:53.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:52 smithi003 ceph-mon[25908]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:53.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: set uid:gid to 167:167 (ceph:ceph) 2024-09-17T09:22:53.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable), process ceph-mon, pid 7 2024-09-17T09:22:53.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: pidfile_write: ignore empty --pid-file 2024-09-17T09:22:53.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: load: jerasure load: lrc 2024-09-17T09:22:53.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: RocksDB version: 7.9.2 2024-09-17T09:22:53.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Git sha 0 2024-09-17T09:22:53.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Compile date 2024-09-16 15:15:13 2024-09-17T09:22:53.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: DB SUMMARY 2024-09-17T09:22:53.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: DB Session ID: 2GEZMKH7JOYVYWFKQWOM 2024-09-17T09:22:53.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: CURRENT file: CURRENT 2024-09-17T09:22:53.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: IDENTITY file: IDENTITY 2024-09-17T09:22:53.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-17T09:22:53.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-17T09:22:53.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-17T09:22:53.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.error_if_exists: 0 2024-09-17T09:22:53.066 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.create_if_missing: 0 2024-09-17T09:22:53.066 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.paranoid_checks: 1 2024-09-17T09:22:53.066 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-17T09:22:53.066 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-17T09:22:53.066 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-17T09:22:53.066 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.env: 0x55a96843dbc0 2024-09-17T09:22:53.066 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.fs: PosixFileSystem 2024-09-17T09:22:53.066 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.info_log: 0x55a968db0be0 2024-09-17T09:22:53.066 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_file_opening_threads: 16 2024-09-17T09:22:53.066 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.statistics: (nil) 2024-09-17T09:22:53.066 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.use_fsync: 0 2024-09-17T09:22:53.066 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_log_file_size: 0 2024-09-17T09:22:53.066 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-17T09:22:53.066 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-17T09:22:53.067 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.keep_log_file_num: 1000 2024-09-17T09:22:53.067 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.recycle_log_file_num: 0 2024-09-17T09:22:53.067 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.allow_fallocate: 1 2024-09-17T09:22:53.067 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.allow_mmap_reads: 0 2024-09-17T09:22:53.067 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.allow_mmap_writes: 0 2024-09-17T09:22:53.067 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.use_direct_reads: 0 2024-09-17T09:22:53.067 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-17T09:22:53.067 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.create_missing_column_families: 0 2024-09-17T09:22:53.067 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.db_log_dir: 2024-09-17T09:22:53.067 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.wal_dir: 2024-09-17T09:22:53.067 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-17T09:22:53.067 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-17T09:22:53.068 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-17T09:22:53.068 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-17T09:22:53.068 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-17T09:22:53.068 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-17T09:22:53.068 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.advise_random_on_open: 1 2024-09-17T09:22:53.068 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.db_write_buffer_size: 0 2024-09-17T09:22:53.068 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.write_buffer_manager: 0x55a968dbe320 2024-09-17T09:22:53.068 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-17T09:22:53.068 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-17T09:22:53.068 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-17T09:22:53.068 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.rate_limiter: (nil) 2024-09-17T09:22:53.068 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-17T09:22:53.069 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.wal_recovery_mode: 2 2024-09-17T09:22:53.069 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.enable_thread_tracking: 0 2024-09-17T09:22:53.069 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.enable_pipelined_write: 0 2024-09-17T09:22:53.069 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.unordered_write: 0 2024-09-17T09:22:53.069 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-17T09:22:53.069 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-17T09:22:53.069 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-17T09:22:53.069 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-17T09:22:53.069 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.row_cache: None 2024-09-17T09:22:53.069 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.wal_filter: None 2024-09-17T09:22:53.069 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-17T09:22:53.069 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.allow_ingest_behind: 0 2024-09-17T09:22:53.069 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.two_write_queues: 0 2024-09-17T09:22:53.069 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.manual_wal_flush: 0 2024-09-17T09:22:53.070 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.wal_compression: 0 2024-09-17T09:22:53.070 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.atomic_flush: 0 2024-09-17T09:22:53.070 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-17T09:22:53.070 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-17T09:22:53.070 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-17T09:22:53.070 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.log_readahead_size: 0 2024-09-17T09:22:53.070 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-17T09:22:53.070 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.best_efforts_recovery: 0 2024-09-17T09:22:53.070 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-17T09:22:53.070 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-17T09:22:53.070 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.allow_data_in_errors: 0 2024-09-17T09:22:53.070 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.db_host_id: __hostname__ 2024-09-17T09:22:53.070 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-17T09:22:53.072 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_background_jobs: 2 2024-09-17T09:22:53.072 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_background_compactions: -1 2024-09-17T09:22:53.072 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_subcompactions: 1 2024-09-17T09:22:53.072 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-17T09:22:53.072 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-17T09:22:53.073 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-17T09:22:53.073 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_total_wal_size: 0 2024-09-17T09:22:53.073 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-17T09:22:53.073 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-17T09:22:53.073 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-17T09:22:53.073 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-17T09:22:53.073 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_open_files: -1 2024-09-17T09:22:53.073 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.bytes_per_sync: 0 2024-09-17T09:22:53.073 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-17T09:22:53.073 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-17T09:22:53.073 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compaction_readahead_size: 0 2024-09-17T09:22:53.073 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_background_flushes: -1 2024-09-17T09:22:53.073 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Compression algorithms supported: 2024-09-17T09:22:53.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: kZSTD supported: 0 2024-09-17T09:22:53.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: kXpressCompression supported: 0 2024-09-17T09:22:53.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: kBZip2Compression supported: 0 2024-09-17T09:22:53.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-17T09:22:53.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: kLZ4Compression supported: 1 2024-09-17T09:22:53.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: kZlibCompression supported: 1 2024-09-17T09:22:53.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: kLZ4HCCompression supported: 1 2024-09-17T09:22:53.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: kSnappyCompression supported: 1 2024-09-17T09:22:53.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-17T09:22:53.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-17T09:22:53.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-17T09:22:53.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-17T09:22:53.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-17T09:22:53.075 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.merge_operator: 2024-09-17T09:22:53.075 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compaction_filter: None 2024-09-17T09:22:53.075 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compaction_filter_factory: None 2024-09-17T09:22:53.075 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.sst_partitioner_factory: None 2024-09-17T09:22:53.075 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-17T09:22:53.075 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-17T09:22:53.075 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a968db07e0) 2024-09-17T09:22:53.075 INFO:journalctl@ceph.mon.c.smithi183.stdout: cache_index_and_filter_blocks: 1 2024-09-17T09:22:53.075 INFO:journalctl@ceph.mon.c.smithi183.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-17T09:22:53.075 INFO:journalctl@ceph.mon.c.smithi183.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-17T09:22:53.075 INFO:journalctl@ceph.mon.c.smithi183.stdout: pin_top_level_index_and_filter: 1 2024-09-17T09:22:53.075 INFO:journalctl@ceph.mon.c.smithi183.stdout: index_type: 0 2024-09-17T09:22:53.075 INFO:journalctl@ceph.mon.c.smithi183.stdout: data_block_index_type: 0 2024-09-17T09:22:53.075 INFO:journalctl@ceph.mon.c.smithi183.stdout: index_shortening: 1 2024-09-17T09:22:53.076 INFO:journalctl@ceph.mon.c.smithi183.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-17T09:22:53.076 INFO:journalctl@ceph.mon.c.smithi183.stdout: checksum: 4 2024-09-17T09:22:53.076 INFO:journalctl@ceph.mon.c.smithi183.stdout: no_block_cache: 0 2024-09-17T09:22:53.076 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache: 0x55a968da5350 2024-09-17T09:22:53.076 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache_name: BinnedLRUCache 2024-09-17T09:22:53.076 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache_options: 2024-09-17T09:22:53.076 INFO:journalctl@ceph.mon.c.smithi183.stdout: capacity : 536870912 2024-09-17T09:22:53.076 INFO:journalctl@ceph.mon.c.smithi183.stdout: num_shard_bits : 4 2024-09-17T09:22:53.076 INFO:journalctl@ceph.mon.c.smithi183.stdout: strict_capacity_limit : 0 2024-09-17T09:22:53.076 INFO:journalctl@ceph.mon.c.smithi183.stdout: high_pri_pool_ratio: 0.000 2024-09-17T09:22:53.076 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache_compressed: (nil) 2024-09-17T09:22:53.076 INFO:journalctl@ceph.mon.c.smithi183.stdout: persistent_cache: (nil) 2024-09-17T09:22:53.076 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_size: 4096 2024-09-17T09:22:53.077 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_size_deviation: 10 2024-09-17T09:22:53.077 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_restart_interval: 16 2024-09-17T09:22:53.077 INFO:journalctl@ceph.mon.c.smithi183.stdout: index_block_restart_interval: 1 2024-09-17T09:22:53.077 INFO:journalctl@ceph.mon.c.smithi183.stdout: metadata_block_size: 4096 2024-09-17T09:22:53.077 INFO:journalctl@ceph.mon.c.smithi183.stdout: partition_filters: 0 2024-09-17T09:22:53.077 INFO:journalctl@ceph.mon.c.smithi183.stdout: use_delta_encoding: 1 2024-09-17T09:22:53.077 INFO:journalctl@ceph.mon.c.smithi183.stdout: filter_policy: bloomfilter 2024-09-17T09:22:53.077 INFO:journalctl@ceph.mon.c.smithi183.stdout: whole_key_filtering: 1 2024-09-17T09:22:53.077 INFO:journalctl@ceph.mon.c.smithi183.stdout: verify_compression: 0 2024-09-17T09:22:53.077 INFO:journalctl@ceph.mon.c.smithi183.stdout: read_amp_bytes_per_bit: 0 2024-09-17T09:22:53.077 INFO:journalctl@ceph.mon.c.smithi183.stdout: format_version: 5 2024-09-17T09:22:53.077 INFO:journalctl@ceph.mon.c.smithi183.stdout: enable_index_compression: 1 2024-09-17T09:22:53.077 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_align: 0 2024-09-17T09:22:53.077 INFO:journalctl@ceph.mon.c.smithi183.stdout: max_auto_readahead_size: 262144 2024-09-17T09:22:53.078 INFO:journalctl@ceph.mon.c.smithi183.stdout: prepopulate_block_cache: 0 2024-09-17T09:22:53.078 INFO:journalctl@ceph.mon.c.smithi183.stdout: initial_auto_readahead_size: 8192 2024-09-17T09:22:53.078 INFO:journalctl@ceph.mon.c.smithi183.stdout: num_file_reads_for_auto_readahead: 2 2024-09-17T09:22:53.078 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.write_buffer_size: 33554432 2024-09-17T09:22:53.078 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_write_buffer_number: 2 2024-09-17T09:22:53.078 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compression: NoCompression 2024-09-17T09:22:53.078 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.bottommost_compression: Disabled 2024-09-17T09:22:53.078 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.prefix_extractor: nullptr 2024-09-17T09:22:53.078 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-17T09:22:53.078 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.num_levels: 7 2024-09-17T09:22:53.078 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-17T09:22:53.078 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-17T09:22:53.078 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-17T09:22:53.078 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-17T09:22:53.079 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-17T09:22:53.079 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-17T09:22:53.079 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-17T09:22:53.079 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-17T09:22:53.079 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-17T09:22:53.079 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-17T09:22:53.079 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-17T09:22:53.079 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-17T09:22:53.079 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-17T09:22:53.079 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compression_opts.level: 32767 2024-09-17T09:22:53.079 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compression_opts.strategy: 0 2024-09-17T09:22:53.079 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-17T09:22:53.079 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-17T09:22:53.080 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-17T09:22:53.080 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-17T09:22:53.080 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compression_opts.enabled: false 2024-09-17T09:22:53.080 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-17T09:22:53.080 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-17T09:22:53.080 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-17T09:22:53.080 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-17T09:22:53.080 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.target_file_size_base: 67108864 2024-09-17T09:22:53.080 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-17T09:22:53.080 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-17T09:22:53.080 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-17T09:22:53.080 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-17T09:22:53.080 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-17T09:22:53.080 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-17T09:22:53.081 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-17T09:22:53.081 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-17T09:22:53.081 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-17T09:22:53.081 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-17T09:22:53.081 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-17T09:22:53.081 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-17T09:22:53.081 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-17T09:22:53.081 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-17T09:22:53.081 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.arena_block_size: 1048576 2024-09-17T09:22:53.081 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-17T09:22:53.081 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-17T09:22:53.081 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.disable_auto_compactions: 0 2024-09-17T09:22:53.081 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-17T09:22:53.082 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-17T09:22:53.082 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-17T09:22:53.082 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-17T09:22:53.082 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-17T09:22:53.082 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-17T09:22:53.082 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-17T09:22:53.082 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-17T09:22:53.082 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-17T09:22:53.082 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-17T09:22:53.082 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-17T09:22:53.082 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.inplace_update_support: 0 2024-09-17T09:22:53.082 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-17T09:22:53.082 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-17T09:22:53.082 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-17T09:22:53.083 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-17T09:22:53.083 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.bloom_locality: 0 2024-09-17T09:22:53.083 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.max_successive_merges: 0 2024-09-17T09:22:53.083 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-17T09:22:53.083 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.paranoid_file_checks: 0 2024-09-17T09:22:53.083 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.force_consistency_checks: 1 2024-09-17T09:22:53.083 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.report_bg_io_stats: 0 2024-09-17T09:22:53.083 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.ttl: 2592000 2024-09-17T09:22:53.083 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-17T09:22:53.083 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-17T09:22:53.083 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-17T09:22:53.083 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.enable_blob_files: false 2024-09-17T09:22:53.083 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.min_blob_size: 0 2024-09-17T09:22:53.083 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.blob_file_size: 268435456 2024-09-17T09:22:53.084 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-17T09:22:53.084 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-17T09:22:53.084 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-17T09:22:53.084 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-17T09:22:53.084 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-17T09:22:53.084 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.blob_file_starting_level: 0 2024-09-17T09:22:53.084 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-17T09:22:53.084 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-17T09:22:53.084 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-17T09:22:53.084 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 98f64200-7080-48e3-8d96-808229ca5f27 2024-09-17T09:22:53.084 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726564972729979, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-17T09:22:53.084 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-17T09:22:53.085 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726564972730375, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726564972, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "98f64200-7080-48e3-8d96-808229ca5f27", "db_session_id": "2GEZMKH7JOYVYWFKQWOM", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-17T09:22:53.085 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726564972730468, "job": 1, "event": "recovery_finished"} 2024-09-17T09:22:53.085 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-17T09:22:53.085 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-17T09:22:53.085 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a968dd0e00 2024-09-17T09:22:53.085 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: DB pointer 0x55a968ec4000 2024-09-17T09:22:53.085 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-17T09:22:53.085 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-17T09:22:53.085 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** DB Stats ** 2024-09-17T09:22:53.085 INFO:journalctl@ceph.mon.c.smithi183.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T09:22:53.085 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-17T09:22:53.085 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T09:22:53.085 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T09:22:53.085 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-17T09:22:53.086 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T09:22:53.086 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T09:22:53.086 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-09-17T09:22:53.086 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** Compaction Stats [default] ** 2024-09-17T09:22:53.086 INFO:journalctl@ceph.mon.c.smithi183.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T09:22:53.086 INFO:journalctl@ceph.mon.c.smithi183.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-17T09:22:53.086 INFO:journalctl@ceph.mon.c.smithi183.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T09:22:53.086 INFO:journalctl@ceph.mon.c.smithi183.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T09:22:53.086 INFO:journalctl@ceph.mon.c.smithi183.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T09:22:53.086 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-09-17T09:22:53.086 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** Compaction Stats [default] ** 2024-09-17T09:22:53.086 INFO:journalctl@ceph.mon.c.smithi183.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T09:22:53.086 INFO:journalctl@ceph.mon.c.smithi183.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-17T09:22:53.086 INFO:journalctl@ceph.mon.c.smithi183.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-09-17T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi183.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-17T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-09-17T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi183.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi183.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-17T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-17T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-17T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-17T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-17T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi183.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-17T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi183.stdout: Block cache BinnedLRUCache@0x55a968da5350#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-17T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi183.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-17T09:22:53.088 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-09-17T09:22:53.088 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-17T09:22:53.088 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-17T09:22:53.088 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: using public_addr v2:172.21.15.183:0/0 -> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] 2024-09-17T09:22:53.088 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: starting mon.c rank -1 at public addrs [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] at bind addrs [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:22:53.088 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: mon.c@-1(???) e0 preinit fsid 08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:22:53.088 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: mon.c@-1(synchronizing).mds e1 new map 2024-09-17T09:22:53.088 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-17T09:22:53.088 INFO:journalctl@ceph.mon.c.smithi183.stdout: e1 2024-09-17T09:22:53.088 INFO:journalctl@ceph.mon.c.smithi183.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T09:22:53.088 INFO:journalctl@ceph.mon.c.smithi183.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T09:22:53.088 INFO:journalctl@ceph.mon.c.smithi183.stdout: legacy client fscid: -1 2024-09-17T09:22:53.088 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-09-17T09:22:53.089 INFO:journalctl@ceph.mon.c.smithi183.stdout: No filesystems configured 2024-09-17T09:22:53.089 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-17T09:22:53.089 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-17T09:22:53.089 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-17T09:22:53.089 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-17T09:22:53.089 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-17T09:22:53.089 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-17T09:22:53.089 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-17T09:22:53.089 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T09:22:53.089 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T09:22:53.089 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T09:22:53.089 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: Updating smithi077:/etc/ceph/ceph.conf 2024-09-17T09:22:53.089 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: Updating smithi077:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:22:53.090 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:22:53.090 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:53.090 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:53.090 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:53.090 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: Updating smithi077:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:22:53.090 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:22:53.090 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:53.090 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: Deploying cephadm binary to smithi183 2024-09-17T09:22:53.090 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:53.090 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:53.090 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:53.090 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: Added host smithi183 2024-09-17T09:22:53.090 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:53.090 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:53.091 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:53.091 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:53.091 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T09:22:53.091 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:53.091 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:53.091 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:53.091 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:53.091 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:53.091 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:22:53.091 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:53.091 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:53.091 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-17T09:22:53.091 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: Updating smithi183:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:22:53.091 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:22:53.092 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: Updating smithi183:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:22:53.092 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/3376471417' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-17T09:22:53.092 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:53.092 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:53.092 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:53.092 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:53.092 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/3376471417' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-17T09:22:53.092 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T09:22:53.092 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:53.092 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi077:172.21.15.77=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:22:53.092 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: Saving service mon spec with placement smithi003:172.21.15.3=a;smithi077:172.21.15.77=b;smithi183:172.21.15.183=c;count:3 2024-09-17T09:22:53.092 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:53.092 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:53.093 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:53.093 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:53.093 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:53.093 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:22:53.093 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:53.093 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: Deploying daemon mon.c on smithi183 2024-09-17T09:22:53.093 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:53.093 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:53.093 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: from='client.? 172.21.15.183:0/2325028841' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T09:22:53.093 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:53.093 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 ceph-mon[30372]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-17T09:22:53.093 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 bash[30333]: d00a181b864177a30934b3e1367ebaa1945b0573d18ee868b53513979a01fcf2 2024-09-17T09:22:53.093 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:52 smithi183 systemd[1]: Started Ceph mon.c for 08f126da-74d6-11ef-bceb-c7b262605968. 2024-09-17T09:22:53.245 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-17T09:22:53.245 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-17T09:22:53.448 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T09:22:58.175 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-17T09:22:58.175 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":2,"fsid":"08f126da-74d6-11ef-bceb-c7b262605968","modified":"2024-09-17T09:22:52.758272Z","created":"2024-09-17T09:21:19.203553Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-17T09:22:58.175 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 2 2024-09-17T09:22:58.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:22:58.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: mon.a calling monitor election 2024-09-17T09:22:58.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:58.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:58.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:58.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:58.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: mon.c calling monitor election 2024-09-17T09:22:58.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:58.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:58.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:58.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:58.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-17T09:22:58.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: monmap e2: 2 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],c=[v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T09:22:58.177 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: fsmap 2024-09-17T09:22:58.177 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T09:22:58.177 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: mgrmap e13: y(active, since 49s) 2024-09-17T09:22:58.177 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: overall HEALTH_OK 2024-09-17T09:22:58.177 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:58.177 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:58.177 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:58.177 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:22:58.177 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:57 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:58.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:22:58.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: mon.a calling monitor election 2024-09-17T09:22:58.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:58.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:58.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:58.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:58.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: mon.c calling monitor election 2024-09-17T09:22:58.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:58.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:58.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:58.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:58.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-17T09:22:58.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: monmap e2: 2 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],c=[v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T09:22:58.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: fsmap 2024-09-17T09:22:58.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T09:22:58.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: mgrmap e13: y(active, since 49s) 2024-09-17T09:22:58.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: overall HEALTH_OK 2024-09-17T09:22:58.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:58.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:58.260 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:22:58.260 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:22:58.260 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:57 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:59.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:58 smithi183 ceph-mon[30372]: Deploying daemon mon.b on smithi077 2024-09-17T09:22:59.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:58 smithi183 ceph-mon[30372]: from='client.? 172.21.15.183:0/4028198983' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T09:22:59.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:58 smithi183 ceph-mon[30372]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:59.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:22:58 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:59.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:58 smithi003 ceph-mon[25908]: Deploying daemon mon.b on smithi077 2024-09-17T09:22:59.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:58 smithi003 ceph-mon[25908]: from='client.? 172.21.15.183:0/4028198983' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T09:22:59.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:58 smithi003 ceph-mon[25908]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:59.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:22:58 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:59.725 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-17T09:22:59.725 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-17T09:22:59.928 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T09:23:00.258 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:22:59 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:22:59.758+0000 7fb5b6031640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-17T09:23:00.563 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:00 smithi183 ceph-mon[30372]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:00.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:00 smithi003 ceph-mon[25908]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:01.176 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 podman[30621]: 2024-09-17 09:23:01.078293509 +0000 UTC m=+0.015694434 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T09:23:01.442 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 podman[30621]: 2024-09-17 09:23:01.390291519 +0000 UTC m=+0.327692435 container init 2f9641ab2c3ab76fbca1c1905ff85e10719940c137d68c97df8dbcab59245177 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b) 2024-09-17T09:23:01.442 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 podman[30621]: 2024-09-17 09:23:01.394577471 +0000 UTC m=+0.331978386 container start 2f9641ab2c3ab76fbca1c1905ff85e10719940c137d68c97df8dbcab59245177 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T09:23:01.442 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: set uid:gid to 167:167 (ceph:ceph) 2024-09-17T09:23:01.442 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable), process ceph-mon, pid 7 2024-09-17T09:23:01.442 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: pidfile_write: ignore empty --pid-file 2024-09-17T09:23:01.442 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: load: jerasure load: lrc 2024-09-17T09:23:01.442 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: RocksDB version: 7.9.2 2024-09-17T09:23:01.442 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Git sha 0 2024-09-17T09:23:01.442 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Compile date 2024-09-16 15:15:13 2024-09-17T09:23:01.442 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: DB SUMMARY 2024-09-17T09:23:01.443 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: DB Session ID: A03F991SDDSUG13IF1HR 2024-09-17T09:23:01.443 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: CURRENT file: CURRENT 2024-09-17T09:23:01.443 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: IDENTITY file: IDENTITY 2024-09-17T09:23:01.443 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-17T09:23:01.443 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-17T09:23:01.443 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-17T09:23:01.443 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.error_if_exists: 0 2024-09-17T09:23:01.443 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.create_if_missing: 0 2024-09-17T09:23:01.443 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.paranoid_checks: 1 2024-09-17T09:23:01.443 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-17T09:23:01.443 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-17T09:23:01.443 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-17T09:23:01.444 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.env: 0x563efb8d0bc0 2024-09-17T09:23:01.444 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.fs: PosixFileSystem 2024-09-17T09:23:01.444 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.info_log: 0x563efd6b4be0 2024-09-17T09:23:01.444 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_file_opening_threads: 16 2024-09-17T09:23:01.444 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.statistics: (nil) 2024-09-17T09:23:01.444 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.use_fsync: 0 2024-09-17T09:23:01.444 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_log_file_size: 0 2024-09-17T09:23:01.444 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-17T09:23:01.444 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-17T09:23:01.444 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.keep_log_file_num: 1000 2024-09-17T09:23:01.444 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.recycle_log_file_num: 0 2024-09-17T09:23:01.444 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.allow_fallocate: 1 2024-09-17T09:23:01.444 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.allow_mmap_reads: 0 2024-09-17T09:23:01.445 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.allow_mmap_writes: 0 2024-09-17T09:23:01.445 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.use_direct_reads: 0 2024-09-17T09:23:01.445 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-17T09:23:01.445 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.create_missing_column_families: 0 2024-09-17T09:23:01.445 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.db_log_dir: 2024-09-17T09:23:01.445 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.wal_dir: 2024-09-17T09:23:01.445 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-17T09:23:01.445 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-17T09:23:01.445 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-17T09:23:01.445 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-17T09:23:01.445 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-17T09:23:01.445 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-17T09:23:01.445 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.advise_random_on_open: 1 2024-09-17T09:23:01.446 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.db_write_buffer_size: 0 2024-09-17T09:23:01.446 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.write_buffer_manager: 0x563efd6c2320 2024-09-17T09:23:01.446 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-17T09:23:01.446 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-17T09:23:01.446 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-17T09:23:01.446 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.rate_limiter: (nil) 2024-09-17T09:23:01.446 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-17T09:23:01.446 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.wal_recovery_mode: 2 2024-09-17T09:23:01.446 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.enable_thread_tracking: 0 2024-09-17T09:23:01.446 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.enable_pipelined_write: 0 2024-09-17T09:23:01.446 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.unordered_write: 0 2024-09-17T09:23:01.446 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-17T09:23:01.446 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-17T09:23:01.447 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-17T09:23:01.447 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-17T09:23:01.447 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.row_cache: None 2024-09-17T09:23:01.447 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.wal_filter: None 2024-09-17T09:23:01.448 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-17T09:23:01.448 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.allow_ingest_behind: 0 2024-09-17T09:23:01.449 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.two_write_queues: 0 2024-09-17T09:23:01.449 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.manual_wal_flush: 0 2024-09-17T09:23:01.449 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.wal_compression: 0 2024-09-17T09:23:01.449 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.atomic_flush: 0 2024-09-17T09:23:01.449 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-17T09:23:01.449 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-17T09:23:01.449 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-17T09:23:01.449 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.log_readahead_size: 0 2024-09-17T09:23:01.449 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-17T09:23:01.449 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.best_efforts_recovery: 0 2024-09-17T09:23:01.449 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-17T09:23:01.449 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-17T09:23:01.449 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.allow_data_in_errors: 0 2024-09-17T09:23:01.450 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.db_host_id: __hostname__ 2024-09-17T09:23:01.450 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-17T09:23:01.450 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_background_jobs: 2 2024-09-17T09:23:01.450 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_background_compactions: -1 2024-09-17T09:23:01.450 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_subcompactions: 1 2024-09-17T09:23:01.450 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-17T09:23:01.450 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-17T09:23:01.450 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-17T09:23:01.450 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_total_wal_size: 0 2024-09-17T09:23:01.450 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-17T09:23:01.450 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-17T09:23:01.450 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-17T09:23:01.450 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-17T09:23:01.451 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_open_files: -1 2024-09-17T09:23:01.451 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.bytes_per_sync: 0 2024-09-17T09:23:01.451 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-17T09:23:01.451 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-17T09:23:01.451 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compaction_readahead_size: 0 2024-09-17T09:23:01.451 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_background_flushes: -1 2024-09-17T09:23:01.451 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Compression algorithms supported: 2024-09-17T09:23:01.451 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: kZSTD supported: 0 2024-09-17T09:23:01.451 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: kXpressCompression supported: 0 2024-09-17T09:23:01.451 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: kBZip2Compression supported: 0 2024-09-17T09:23:01.451 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-17T09:23:01.451 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: kLZ4Compression supported: 1 2024-09-17T09:23:01.451 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: kZlibCompression supported: 1 2024-09-17T09:23:01.451 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: kLZ4HCCompression supported: 1 2024-09-17T09:23:01.452 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: kSnappyCompression supported: 1 2024-09-17T09:23:01.452 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-17T09:23:01.452 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-17T09:23:01.452 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-17T09:23:01.452 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-17T09:23:01.452 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-17T09:23:01.452 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.merge_operator: 2024-09-17T09:23:01.452 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compaction_filter: None 2024-09-17T09:23:01.452 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compaction_filter_factory: None 2024-09-17T09:23:01.452 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.sst_partitioner_factory: None 2024-09-17T09:23:01.452 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-17T09:23:01.452 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-17T09:23:01.452 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563efd6b47e0) 2024-09-17T09:23:01.453 INFO:journalctl@ceph.mon.b.smithi077.stdout: cache_index_and_filter_blocks: 1 2024-09-17T09:23:01.453 INFO:journalctl@ceph.mon.b.smithi077.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-17T09:23:01.453 INFO:journalctl@ceph.mon.b.smithi077.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-17T09:23:01.453 INFO:journalctl@ceph.mon.b.smithi077.stdout: pin_top_level_index_and_filter: 1 2024-09-17T09:23:01.453 INFO:journalctl@ceph.mon.b.smithi077.stdout: index_type: 0 2024-09-17T09:23:01.453 INFO:journalctl@ceph.mon.b.smithi077.stdout: data_block_index_type: 0 2024-09-17T09:23:01.453 INFO:journalctl@ceph.mon.b.smithi077.stdout: index_shortening: 1 2024-09-17T09:23:01.453 INFO:journalctl@ceph.mon.b.smithi077.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-17T09:23:01.453 INFO:journalctl@ceph.mon.b.smithi077.stdout: checksum: 4 2024-09-17T09:23:01.453 INFO:journalctl@ceph.mon.b.smithi077.stdout: no_block_cache: 0 2024-09-17T09:23:01.453 INFO:journalctl@ceph.mon.b.smithi077.stdout: block_cache: 0x563efd6a9350 2024-09-17T09:23:01.453 INFO:journalctl@ceph.mon.b.smithi077.stdout: block_cache_name: BinnedLRUCache 2024-09-17T09:23:01.453 INFO:journalctl@ceph.mon.b.smithi077.stdout: block_cache_options: 2024-09-17T09:23:01.453 INFO:journalctl@ceph.mon.b.smithi077.stdout: capacity : 536870912 2024-09-17T09:23:01.454 INFO:journalctl@ceph.mon.b.smithi077.stdout: num_shard_bits : 4 2024-09-17T09:23:01.454 INFO:journalctl@ceph.mon.b.smithi077.stdout: strict_capacity_limit : 0 2024-09-17T09:23:01.454 INFO:journalctl@ceph.mon.b.smithi077.stdout: high_pri_pool_ratio: 0.000 2024-09-17T09:23:01.454 INFO:journalctl@ceph.mon.b.smithi077.stdout: block_cache_compressed: (nil) 2024-09-17T09:23:01.454 INFO:journalctl@ceph.mon.b.smithi077.stdout: persistent_cache: (nil) 2024-09-17T09:23:01.454 INFO:journalctl@ceph.mon.b.smithi077.stdout: block_size: 4096 2024-09-17T09:23:01.454 INFO:journalctl@ceph.mon.b.smithi077.stdout: block_size_deviation: 10 2024-09-17T09:23:01.454 INFO:journalctl@ceph.mon.b.smithi077.stdout: block_restart_interval: 16 2024-09-17T09:23:01.454 INFO:journalctl@ceph.mon.b.smithi077.stdout: index_block_restart_interval: 1 2024-09-17T09:23:01.454 INFO:journalctl@ceph.mon.b.smithi077.stdout: metadata_block_size: 4096 2024-09-17T09:23:01.454 INFO:journalctl@ceph.mon.b.smithi077.stdout: partition_filters: 0 2024-09-17T09:23:01.454 INFO:journalctl@ceph.mon.b.smithi077.stdout: use_delta_encoding: 1 2024-09-17T09:23:01.454 INFO:journalctl@ceph.mon.b.smithi077.stdout: filter_policy: bloomfilter 2024-09-17T09:23:01.454 INFO:journalctl@ceph.mon.b.smithi077.stdout: whole_key_filtering: 1 2024-09-17T09:23:01.455 INFO:journalctl@ceph.mon.b.smithi077.stdout: verify_compression: 0 2024-09-17T09:23:01.455 INFO:journalctl@ceph.mon.b.smithi077.stdout: read_amp_bytes_per_bit: 0 2024-09-17T09:23:01.455 INFO:journalctl@ceph.mon.b.smithi077.stdout: format_version: 5 2024-09-17T09:23:01.455 INFO:journalctl@ceph.mon.b.smithi077.stdout: enable_index_compression: 1 2024-09-17T09:23:01.455 INFO:journalctl@ceph.mon.b.smithi077.stdout: block_align: 0 2024-09-17T09:23:01.455 INFO:journalctl@ceph.mon.b.smithi077.stdout: max_auto_readahead_size: 262144 2024-09-17T09:23:01.455 INFO:journalctl@ceph.mon.b.smithi077.stdout: prepopulate_block_cache: 0 2024-09-17T09:23:01.455 INFO:journalctl@ceph.mon.b.smithi077.stdout: initial_auto_readahead_size: 8192 2024-09-17T09:23:01.455 INFO:journalctl@ceph.mon.b.smithi077.stdout: num_file_reads_for_auto_readahead: 2 2024-09-17T09:23:01.455 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.write_buffer_size: 33554432 2024-09-17T09:23:01.455 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_write_buffer_number: 2 2024-09-17T09:23:01.455 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compression: NoCompression 2024-09-17T09:23:01.455 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.bottommost_compression: Disabled 2024-09-17T09:23:01.455 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.prefix_extractor: nullptr 2024-09-17T09:23:01.456 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-17T09:23:01.456 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.num_levels: 7 2024-09-17T09:23:01.456 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-17T09:23:01.456 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-17T09:23:01.456 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-17T09:23:01.456 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-17T09:23:01.456 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-17T09:23:01.456 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-17T09:23:01.456 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-17T09:23:01.456 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-17T09:23:01.456 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-17T09:23:01.456 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-17T09:23:01.456 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-17T09:23:01.457 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-17T09:23:01.457 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-17T09:23:01.457 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compression_opts.level: 32767 2024-09-17T09:23:01.457 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compression_opts.strategy: 0 2024-09-17T09:23:01.457 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-17T09:23:01.457 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-17T09:23:01.457 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-17T09:23:01.457 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-17T09:23:01.457 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compression_opts.enabled: false 2024-09-17T09:23:01.457 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-17T09:23:01.457 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-17T09:23:01.457 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-17T09:23:01.457 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-17T09:23:01.457 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.target_file_size_base: 67108864 2024-09-17T09:23:01.458 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-17T09:23:01.458 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-17T09:23:01.458 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-17T09:23:01.458 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-17T09:23:01.458 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-17T09:23:01.458 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-17T09:23:01.458 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-17T09:23:01.458 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-17T09:23:01.458 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-17T09:23:01.458 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-17T09:23:01.458 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-17T09:23:01.458 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-17T09:23:01.458 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-17T09:23:01.459 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-17T09:23:01.459 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.arena_block_size: 1048576 2024-09-17T09:23:01.459 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-17T09:23:01.459 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-17T09:23:01.459 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.disable_auto_compactions: 0 2024-09-17T09:23:01.459 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-17T09:23:01.459 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-17T09:23:01.459 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-17T09:23:01.459 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-17T09:23:01.459 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-17T09:23:01.459 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-17T09:23:01.460 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-17T09:23:01.460 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-17T09:23:01.460 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-17T09:23:01.460 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-17T09:23:01.460 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-17T09:23:01.460 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.inplace_update_support: 0 2024-09-17T09:23:01.460 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-17T09:23:01.460 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-17T09:23:01.460 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-17T09:23:01.460 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-17T09:23:01.460 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.bloom_locality: 0 2024-09-17T09:23:01.460 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.max_successive_merges: 0 2024-09-17T09:23:01.460 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-17T09:23:01.461 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.paranoid_file_checks: 0 2024-09-17T09:23:01.461 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.force_consistency_checks: 1 2024-09-17T09:23:01.461 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.report_bg_io_stats: 0 2024-09-17T09:23:01.461 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.ttl: 2592000 2024-09-17T09:23:01.461 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-17T09:23:01.461 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-17T09:23:01.461 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-17T09:23:01.461 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.enable_blob_files: false 2024-09-17T09:23:01.461 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.min_blob_size: 0 2024-09-17T09:23:01.461 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.blob_file_size: 268435456 2024-09-17T09:23:01.461 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-17T09:23:01.461 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-17T09:23:01.461 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-17T09:23:01.462 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-17T09:23:01.462 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-17T09:23:01.462 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.blob_file_starting_level: 0 2024-09-17T09:23:01.462 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-17T09:23:01.462 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-17T09:23:01.462 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-17T09:23:01.462 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9f664f30-01a7-4c49-b734-ae84455e45c6 2024-09-17T09:23:01.462 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726564981421444, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-17T09:23:01.462 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-17T09:23:01.462 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726564981421839, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726564981, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9f664f30-01a7-4c49-b734-ae84455e45c6", "db_session_id": "A03F991SDDSUG13IF1HR", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-17T09:23:01.462 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726564981421936, "job": 1, "event": "recovery_finished"} 2024-09-17T09:23:01.462 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-17T09:23:01.462 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-17T09:23:01.462 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563efd6d4e00 2024-09-17T09:23:01.463 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: DB pointer 0x563efd7c8000 2024-09-17T09:23:01.463 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-17T09:23:01.463 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-17T09:23:01.463 INFO:journalctl@ceph.mon.b.smithi077.stdout: ** DB Stats ** 2024-09-17T09:23:01.463 INFO:journalctl@ceph.mon.b.smithi077.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T09:23:01.463 INFO:journalctl@ceph.mon.b.smithi077.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-17T09:23:01.463 INFO:journalctl@ceph.mon.b.smithi077.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T09:23:01.463 INFO:journalctl@ceph.mon.b.smithi077.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T09:23:01.463 INFO:journalctl@ceph.mon.b.smithi077.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-17T09:23:01.463 INFO:journalctl@ceph.mon.b.smithi077.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T09:23:01.463 INFO:journalctl@ceph.mon.b.smithi077.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T09:23:01.463 INFO:journalctl@ceph.mon.b.smithi077.stdout: 2024-09-17T09:23:01.463 INFO:journalctl@ceph.mon.b.smithi077.stdout: ** Compaction Stats [default] ** 2024-09-17T09:23:01.463 INFO:journalctl@ceph.mon.b.smithi077.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T09:23:01.464 INFO:journalctl@ceph.mon.b.smithi077.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-17T09:23:01.464 INFO:journalctl@ceph.mon.b.smithi077.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T09:23:01.464 INFO:journalctl@ceph.mon.b.smithi077.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T09:23:01.464 INFO:journalctl@ceph.mon.b.smithi077.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T09:23:01.464 INFO:journalctl@ceph.mon.b.smithi077.stdout: 2024-09-17T09:23:01.464 INFO:journalctl@ceph.mon.b.smithi077.stdout: ** Compaction Stats [default] ** 2024-09-17T09:23:01.464 INFO:journalctl@ceph.mon.b.smithi077.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T09:23:01.464 INFO:journalctl@ceph.mon.b.smithi077.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-17T09:23:01.464 INFO:journalctl@ceph.mon.b.smithi077.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T09:23:01.464 INFO:journalctl@ceph.mon.b.smithi077.stdout: 2024-09-17T09:23:01.464 INFO:journalctl@ceph.mon.b.smithi077.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-17T09:23:01.464 INFO:journalctl@ceph.mon.b.smithi077.stdout: 2024-09-17T09:23:01.464 INFO:journalctl@ceph.mon.b.smithi077.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T09:23:01.465 INFO:journalctl@ceph.mon.b.smithi077.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-17T09:23:01.465 INFO:journalctl@ceph.mon.b.smithi077.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-17T09:23:01.465 INFO:journalctl@ceph.mon.b.smithi077.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-17T09:23:01.465 INFO:journalctl@ceph.mon.b.smithi077.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-17T09:23:01.465 INFO:journalctl@ceph.mon.b.smithi077.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-17T09:23:01.465 INFO:journalctl@ceph.mon.b.smithi077.stdout: Cumulative compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T09:23:01.465 INFO:journalctl@ceph.mon.b.smithi077.stdout: Interval compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T09:23:01.465 INFO:journalctl@ceph.mon.b.smithi077.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-17T09:23:01.465 INFO:journalctl@ceph.mon.b.smithi077.stdout: Block cache BinnedLRUCache@0x563efd6a9350#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-17T09:23:01.465 INFO:journalctl@ceph.mon.b.smithi077.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-17T09:23:01.465 INFO:journalctl@ceph.mon.b.smithi077.stdout: 2024-09-17T09:23:01.465 INFO:journalctl@ceph.mon.b.smithi077.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-17T09:23:01.465 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-17T09:23:01.465 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: using public_addr v2:172.21.15.77:0/0 -> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] 2024-09-17T09:23:01.466 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: starting mon.b rank -1 at public addrs [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] at bind addrs [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:23:01.466 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: mon.b@-1(???) e0 preinit fsid 08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:23:01.466 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: mon.b@-1(synchronizing).mds e1 new map 2024-09-17T09:23:01.466 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-17T09:23:01.466 INFO:journalctl@ceph.mon.b.smithi077.stdout: e1 2024-09-17T09:23:01.466 INFO:journalctl@ceph.mon.b.smithi077.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T09:23:01.466 INFO:journalctl@ceph.mon.b.smithi077.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T09:23:01.466 INFO:journalctl@ceph.mon.b.smithi077.stdout: legacy client fscid: -1 2024-09-17T09:23:01.466 INFO:journalctl@ceph.mon.b.smithi077.stdout: 2024-09-17T09:23:01.466 INFO:journalctl@ceph.mon.b.smithi077.stdout: No filesystems configured 2024-09-17T09:23:01.466 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-17T09:23:01.466 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-17T09:23:01.466 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-17T09:23:01.467 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-17T09:23:01.467 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-17T09:23:01.467 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-17T09:23:01.467 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-17T09:23:01.467 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T09:23:01.467 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T09:23:01.467 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T09:23:01.467 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: Updating smithi077:/etc/ceph/ceph.conf 2024-09-17T09:23:01.467 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: Updating smithi077:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:23:01.467 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:23:01.467 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:01.467 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:01.467 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:01.467 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: Updating smithi077:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:23:01.468 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:23:01.468 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:01.468 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: Deploying cephadm binary to smithi183 2024-09-17T09:23:01.468 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:01.468 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:01.468 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:01.468 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: Added host smithi183 2024-09-17T09:23:01.468 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:01.468 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:01.468 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:01.468 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:01.468 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T09:23:01.468 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:01.469 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:01.469 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:01.469 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:01.469 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:01.469 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:01.469 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:01.469 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:01.469 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-17T09:23:01.469 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: Updating smithi183:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:23:01.469 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:23:01.469 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: Updating smithi183:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:23:01.469 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/3376471417' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-17T09:23:01.469 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:01.469 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:01.470 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:01.470 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:01.470 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/3376471417' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-17T09:23:01.470 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T09:23:01.470 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:01.470 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi077:172.21.15.77=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:23:01.470 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: Saving service mon spec with placement smithi003:172.21.15.3=a;smithi077:172.21.15.77=b;smithi183:172.21.15.183=c;count:3 2024-09-17T09:23:01.470 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:01.470 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:01.470 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:01.470 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:01.470 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:01.470 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:23:01.470 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:01.471 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: Deploying daemon mon.c on smithi183 2024-09-17T09:23:01.471 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:01.471 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:01.471 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='client.? 172.21.15.183:0/2325028841' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T09:23:01.471 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:01.471 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:23:01.471 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: mon.a calling monitor election 2024-09-17T09:23:01.471 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:23:01.471 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:23:01.471 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:01.471 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:23:01.471 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: mon.c calling monitor election 2024-09-17T09:23:01.471 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:23:01.471 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:01.472 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:23:01.472 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:23:01.472 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-17T09:23:01.472 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: monmap e2: 2 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],c=[v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T09:23:01.472 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: fsmap 2024-09-17T09:23:01.472 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T09:23:01.472 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: mgrmap e13: y(active, since 49s) 2024-09-17T09:23:01.472 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: overall HEALTH_OK 2024-09-17T09:23:01.472 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:01.472 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:01.472 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:01.472 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:23:01.472 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:01.473 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: Deploying daemon mon.b on smithi077 2024-09-17T09:23:01.473 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='client.? 172.21.15.183:0/4028198983' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T09:23:01.473 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:01.473 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:23:01.473 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:01.473 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 ceph-mon[30659]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-17T09:23:01.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 bash[30621]: 2f9641ab2c3ab76fbca1c1905ff85e10719940c137d68c97df8dbcab59245177 2024-09-17T09:23:01.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:01 smithi077 systemd[1]: Started Ceph mon.b for 08f126da-74d6-11ef-bceb-c7b262605968. 2024-09-17T09:23:06.456 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-17T09:23:06.457 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":3,"fsid":"08f126da-74d6-11ef-bceb-c7b262605968","modified":"2024-09-17T09:23:01.446059Z","created":"2024-09-17T09:21:19.203553Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-17T09:23:06.457 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 3 2024-09-17T09:23:06.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:23:06.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: mon.c calling monitor election 2024-09-17T09:23:06.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: mon.a calling monitor election 2024-09-17T09:23:06.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:06.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:23:06.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: from='client.? 172.21.15.183:0/1565412532' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T09:23:06.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:06.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:06.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:06.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:06.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:06.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:06.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:06.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:06.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-17T09:23:06.760 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: monmap e3: 3 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],b=[v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0],c=[v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T09:23:06.760 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: fsmap 2024-09-17T09:23:06.760 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T09:23:06.760 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: mgrmap e13: y(active, since 58s) 2024-09-17T09:23:06.760 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-09-17T09:23:06.760 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-09-17T09:23:06.760 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-09-17T09:23:06.760 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: mon.b (rank 2) addr [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] is down (out of quorum) 2024-09-17T09:23:06.760 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:06.760 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:06.760 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:06.760 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:06 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:06.806 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:23:06.806 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: mon.c calling monitor election 2024-09-17T09:23:06.806 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: mon.a calling monitor election 2024-09-17T09:23:06.806 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:06.807 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:23:06.807 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: from='client.? 172.21.15.183:0/1565412532' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T09:23:06.807 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:06.807 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:06.807 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:06.807 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:06.807 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:06.807 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:06.807 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:06.807 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:06.807 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-17T09:23:06.807 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: monmap e3: 3 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],b=[v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0],c=[v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T09:23:06.807 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: fsmap 2024-09-17T09:23:06.808 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T09:23:06.808 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: mgrmap e13: y(active, since 58s) 2024-09-17T09:23:06.808 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-09-17T09:23:06.808 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-09-17T09:23:06.808 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-09-17T09:23:06.808 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: mon.b (rank 2) addr [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] is down (out of quorum) 2024-09-17T09:23:06.808 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:06.808 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:06.808 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:06.808 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:06 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:07.110 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-17T09:23:07.110 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-17T09:23:07.320 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:23:07.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:07 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:07.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:07 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:07.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:07 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:07.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:07 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:08.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:23:08.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: mon.c calling monitor election 2024-09-17T09:23:08.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: mon.a calling monitor election 2024-09-17T09:23:08.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:08.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:23:08.756 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: from='client.? 172.21.15.183:0/1565412532' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T09:23:08.757 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:08.757 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:08.757 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:08.757 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:08.757 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:08.757 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:08.757 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:08.757 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:08.757 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-17T09:23:08.757 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: monmap e3: 3 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],b=[v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0],c=[v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T09:23:08.757 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: fsmap 2024-09-17T09:23:08.757 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T09:23:08.757 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: mgrmap e13: y(active, since 58s) 2024-09-17T09:23:08.758 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-09-17T09:23:08.758 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-09-17T09:23:08.758 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-09-17T09:23:08.758 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: mon.b (rank 2) addr [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] is down (out of quorum) 2024-09-17T09:23:08.758 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:08.758 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:08.758 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:08.758 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:08.758 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:08.758 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:08 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:09.161 INFO:teuthology.orchestra.run.smithi003.stdout:# minimal ceph.conf for 08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:23:09.161 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-09-17T09:23:09.161 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:23:09.161 INFO:teuthology.orchestra.run.smithi003.stdout: mon_host = [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] 2024-09-17T09:23:09.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: mon.b calling monitor election 2024-09-17T09:23:09.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: mon.b calling monitor election 2024-09-17T09:23:09.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: mon.c calling monitor election 2024-09-17T09:23:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: mon.a calling monitor election 2024-09-17T09:23:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-17T09:23:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: monmap e3: 3 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],b=[v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0],c=[v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T09:23:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: fsmap 2024-09-17T09:23:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T09:23:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: mgrmap e13: y(active, since 60s) 2024-09-17T09:23:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-09-17T09:23:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: Cluster is now healthy 2024-09-17T09:23:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: overall HEALTH_OK 2024-09-17T09:23:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: Reconfiguring mon.a (unknown last config time)... 2024-09-17T09:23:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:23:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T09:23:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: Reconfiguring daemon mon.a on smithi003 2024-09-17T09:23:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:09 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/3726796308' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:09.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: mon.b calling monitor election 2024-09-17T09:23:09.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: mon.b calling monitor election 2024-09-17T09:23:09.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: mon.c calling monitor election 2024-09-17T09:23:09.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: mon.a calling monitor election 2024-09-17T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-17T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: monmap e3: 3 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],b=[v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0],c=[v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: fsmap 2024-09-17T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: mgrmap e13: y(active, since 60s) 2024-09-17T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-09-17T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: Cluster is now healthy 2024-09-17T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: overall HEALTH_OK 2024-09-17T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: Reconfiguring mon.a (unknown last config time)... 2024-09-17T09:23:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:23:09.754 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T09:23:09.754 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:09.754 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: Reconfiguring daemon mon.a on smithi003 2024-09-17T09:23:09.754 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:09 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/3726796308' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:09.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: mon.b calling monitor election 2024-09-17T09:23:09.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: mon.b calling monitor election 2024-09-17T09:23:09.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: mon.c calling monitor election 2024-09-17T09:23:09.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: mon.a calling monitor election 2024-09-17T09:23:09.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-17T09:23:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: monmap e3: 3 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],b=[v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0],c=[v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T09:23:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: fsmap 2024-09-17T09:23:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T09:23:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: mgrmap e13: y(active, since 60s) 2024-09-17T09:23:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-09-17T09:23:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: Cluster is now healthy 2024-09-17T09:23:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: overall HEALTH_OK 2024-09-17T09:23:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: Reconfiguring mon.a (unknown last config time)... 2024-09-17T09:23:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:23:09.816 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T09:23:09.816 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:09.816 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: Reconfiguring daemon mon.a on smithi003 2024-09-17T09:23:09.816 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:09 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/3726796308' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:09.848 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-17T09:23:09.849 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-17T09:23:09.849 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.conf 2024-09-17T09:23:09.877 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-17T09:23:09.877 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:23:09.944 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-17T09:23:09.944 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.conf 2024-09-17T09:23:09.973 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-17T09:23:09.973 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:23:10.039 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-17T09:23:10.039 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.conf 2024-09-17T09:23:10.072 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-17T09:23:10.073 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:23:10.142 INFO:tasks.cephadm:Adding mgr.y on smithi003 2024-09-17T09:23:10.142 INFO:tasks.cephadm:Adding mgr.x on smithi077 2024-09-17T09:23:10.143 INFO:tasks.cephadm:Adding mgr.z on smithi183 2024-09-17T09:23:10.143 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi003=y;smithi077=x;smithi183=z' 2024-09-17T09:23:10.373 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T09:23:10.721 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:10 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:10.721 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:10 smithi183 ceph-mon[30372]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:10.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:10 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:10.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:10 smithi003 ceph-mon[25908]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:10.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:10 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:23:10.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:10 smithi077 ceph-mon[30659]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:12.050 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:11 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:12.050 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:11 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:12.050 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:11 smithi077 ceph-mon[30659]: Reconfiguring mon.b (monmap changed)... 2024-09-17T09:23:12.050 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:11 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:23:12.050 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:11 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T09:23:12.050 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:11 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:12.050 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:11 smithi077 ceph-mon[30659]: Reconfiguring daemon mon.b on smithi077 2024-09-17T09:23:12.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:11 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:12.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:11 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:12.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:11 smithi183 ceph-mon[30372]: Reconfiguring mon.b (monmap changed)... 2024-09-17T09:23:12.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:11 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:23:12.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:11 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T09:23:12.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:11 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:12.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:11 smithi183 ceph-mon[30372]: Reconfiguring daemon mon.b on smithi077 2024-09-17T09:23:12.085 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mgr update... 2024-09-17T09:23:12.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:11 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:12.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:11 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:12.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:11 smithi003 ceph-mon[25908]: Reconfiguring mon.b (monmap changed)... 2024-09-17T09:23:12.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:11 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:23:12.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:11 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T09:23:12.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:11 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:12.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:11 smithi003 ceph-mon[25908]: Reconfiguring daemon mon.b on smithi077 2024-09-17T09:23:12.702 DEBUG:teuthology.orchestra.run.smithi077:mgr.x> sudo journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@mgr.x.service 2024-09-17T09:23:12.705 DEBUG:teuthology.orchestra.run.smithi183:mgr.z> sudo journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@mgr.z.service 2024-09-17T09:23:12.707 INFO:tasks.cephadm:Deploying OSDs... 2024-09-17T09:23:12.707 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-17T09:23:12.707 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T09:23:12.727 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-17T09:23:12.728 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/nvme4n1 2024-09-17T09:23:12.787 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/nvme4n1 2024-09-17T09:23:12.787 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T09:23:12.787 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 1040 Links: 1 Device type: 103,8 2024-09-17T09:23:12.787 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T09:23:12.787 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T09:23:12.787 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-17 09:22:20.612552135 +0000 2024-09-17T09:23:12.787 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-17 09:20:05.898124543 +0000 2024-09-17T09:23:12.787 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-17 09:20:05.898124543 +0000 2024-09-17T09:23:12.788 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-17 09:20:05.894124589 +0000 2024-09-17T09:23:12.788 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-17T09:23:12.858 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-17T09:23:12.858 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-17T09:23:12.858 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000239182 s, 2.1 MB/s 2024-09-17T09:23:12.860 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-17T09:23:12.920 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/nvme3n1 2024-09-17T09:23:12.978 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/nvme3n1 2024-09-17T09:23:12.978 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T09:23:12.979 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 1028 Links: 1 Device type: 103,6 2024-09-17T09:23:12.979 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T09:23:12.979 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T09:23:12.979 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-17 09:22:20.601552265 +0000 2024-09-17T09:23:12.979 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-17 09:20:05.755126201 +0000 2024-09-17T09:23:12.979 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-17 09:20:05.755126201 +0000 2024-09-17T09:23:12.979 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-17 09:20:05.750126259 +0000 2024-09-17T09:23:12.979 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-17T09:23:13.046 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-17T09:23:13.047 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-17T09:23:13.047 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000253841 s, 2.0 MB/s 2024-09-17T09:23:13.048 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-17T09:23:13.108 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/nvme2n1 2024-09-17T09:23:13.166 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/nvme2n1 2024-09-17T09:23:13.166 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T09:23:13.166 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 1016 Links: 1 Device type: 103,4 2024-09-17T09:23:13.166 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T09:23:13.166 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T09:23:13.166 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-17 09:22:20.591552384 +0000 2024-09-17T09:23:13.167 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-17 09:20:05.614127835 +0000 2024-09-17T09:23:13.167 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-17 09:20:05.614127835 +0000 2024-09-17T09:23:13.167 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-17 09:20:05.610127882 +0000 2024-09-17T09:23:13.167 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-17T09:23:13.200 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:13 smithi183 ceph-mon[30372]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=y;smithi077=x;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:23:13.200 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:13 smithi183 ceph-mon[30372]: Saving service mgr spec with placement smithi003=y;smithi077=x;smithi183=z;count:3 2024-09-17T09:23:13.200 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:13 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:13.200 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:13 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:13.200 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:13 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:13.200 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:13 smithi183 ceph-mon[30372]: Reconfiguring mon.c (monmap changed)... 2024-09-17T09:23:13.200 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:13 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:23:13.200 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:13 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T09:23:13.201 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:13 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:13.201 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:13 smithi183 ceph-mon[30372]: Reconfiguring daemon mon.c on smithi183 2024-09-17T09:23:13.201 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:13 smithi183 ceph-mon[30372]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:13.230 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:13 smithi003 ceph-mon[25908]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=y;smithi077=x;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:23:13.230 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:13 smithi003 ceph-mon[25908]: Saving service mgr spec with placement smithi003=y;smithi077=x;smithi183=z;count:3 2024-09-17T09:23:13.230 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:13 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:13.230 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:13 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:13.230 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:13 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:13.230 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:13 smithi003 ceph-mon[25908]: Reconfiguring mon.c (monmap changed)... 2024-09-17T09:23:13.230 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:13 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:23:13.230 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:13 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T09:23:13.231 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:13 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:13.231 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:13 smithi003 ceph-mon[25908]: Reconfiguring daemon mon.c on smithi183 2024-09-17T09:23:13.231 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:13 smithi003 ceph-mon[25908]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:13.232 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-17T09:23:13.232 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-17T09:23:13.232 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000217509 s, 2.4 MB/s 2024-09-17T09:23:13.233 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-17T09:23:13.291 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/nvme1n1 2024-09-17T09:23:13.348 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/nvme1n1 2024-09-17T09:23:13.349 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T09:23:13.349 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 1004 Links: 1 Device type: 103,2 2024-09-17T09:23:13.349 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T09:23:13.349 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T09:23:13.349 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-17 09:22:20.580552514 +0000 2024-09-17T09:23:13.349 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-17 09:20:05.475129447 +0000 2024-09-17T09:23:13.349 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-17 09:20:05.475129447 +0000 2024-09-17T09:23:13.350 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-17 09:20:05.469129516 +0000 2024-09-17T09:23:13.350 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-17T09:23:13.416 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-17T09:23:13.416 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-17T09:23:13.416 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000382692 s, 1.3 MB/s 2024-09-17T09:23:13.417 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-17T09:23:13.476 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-17T09:23:13.476 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T09:23:13.496 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-17T09:23:13.497 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/nvme4n1 2024-09-17T09:23:13.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:13 smithi077 ceph-mon[30659]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=y;smithi077=x;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:23:13.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:13 smithi077 ceph-mon[30659]: Saving service mgr spec with placement smithi003=y;smithi077=x;smithi183=z;count:3 2024-09-17T09:23:13.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:13 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:13.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:13 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:13.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:13 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:13.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:13 smithi077 ceph-mon[30659]: Reconfiguring mon.c (monmap changed)... 2024-09-17T09:23:13.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:13 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:23:13.503 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:13 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T09:23:13.503 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:13 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:13.503 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:13 smithi077 ceph-mon[30659]: Reconfiguring daemon mon.c on smithi183 2024-09-17T09:23:13.503 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:13 smithi077 ceph-mon[30659]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:13.520 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/nvme4n1 2024-09-17T09:23:13.520 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T09:23:13.520 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 1040 Links: 1 Device type: 103,8 2024-09-17T09:23:13.520 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T09:23:13.520 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T09:23:13.520 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-09-17 09:22:33.732476155 +0000 2024-09-17T09:23:13.520 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-09-17 09:20:07.501564820 +0000 2024-09-17T09:23:13.520 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-09-17 09:20:07.501564820 +0000 2024-09-17T09:23:13.520 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-09-17 09:20:07.496564927 +0000 2024-09-17T09:23:13.521 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-17T09:23:13.587 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-09-17T09:23:13.587 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-09-17T09:23:13.587 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000235216 s, 2.2 MB/s 2024-09-17T09:23:13.588 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-17T09:23:13.647 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/nvme3n1 2024-09-17T09:23:13.703 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/nvme3n1 2024-09-17T09:23:13.704 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T09:23:13.704 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 1028 Links: 1 Device type: 103,6 2024-09-17T09:23:13.704 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T09:23:13.704 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T09:23:13.704 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-09-17 09:22:33.722476364 +0000 2024-09-17T09:23:13.704 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-09-17 09:20:07.357567906 +0000 2024-09-17T09:23:13.704 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-09-17 09:20:07.357567906 +0000 2024-09-17T09:23:13.704 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-09-17 09:20:07.353567992 +0000 2024-09-17T09:23:13.704 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-17T09:23:13.770 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-09-17T09:23:13.770 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-09-17T09:23:13.770 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000252033 s, 2.0 MB/s 2024-09-17T09:23:13.772 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-17T09:23:13.829 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/nvme2n1 2024-09-17T09:23:13.886 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/nvme2n1 2024-09-17T09:23:13.886 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T09:23:13.886 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 1015 Links: 1 Device type: 103,4 2024-09-17T09:23:13.886 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T09:23:13.887 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T09:23:13.887 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-09-17 09:22:33.712476573 +0000 2024-09-17T09:23:13.887 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-09-17 09:20:07.224570757 +0000 2024-09-17T09:23:13.887 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-09-17 09:20:07.224570757 +0000 2024-09-17T09:23:13.887 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-09-17 09:20:07.219570864 +0000 2024-09-17T09:23:13.887 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-17T09:23:13.952 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-09-17T09:23:13.952 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-09-17T09:23:13.952 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000238583 s, 2.1 MB/s 2024-09-17T09:23:13.953 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-17T09:23:14.010 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/nvme1n1 2024-09-17T09:23:14.066 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/nvme1n1 2024-09-17T09:23:14.066 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T09:23:14.066 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 1004 Links: 1 Device type: 103,2 2024-09-17T09:23:14.066 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T09:23:14.066 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T09:23:14.066 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-09-17 09:22:33.702476782 +0000 2024-09-17T09:23:14.066 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-09-17 09:20:07.084573757 +0000 2024-09-17T09:23:14.066 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-09-17 09:20:07.084573757 +0000 2024-09-17T09:23:14.066 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-09-17 09:20:07.079573864 +0000 2024-09-17T09:23:14.067 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-17T09:23:14.129 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-09-17T09:23:14.129 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-09-17T09:23:14.129 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000234358 s, 2.2 MB/s 2024-09-17T09:23:14.131 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-17T09:23:14.187 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-17T09:23:14.187 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T09:23:14.203 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-17T09:23:14.204 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/nvme4n1 2024-09-17T09:23:14.261 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/nvme4n1 2024-09-17T09:23:14.261 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T09:23:14.261 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 1039 Links: 1 Device type: 103,8 2024-09-17T09:23:14.261 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T09:23:14.261 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T09:23:14.261 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-17 09:22:42.404890154 +0000 2024-09-17T09:23:14.261 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-17 09:20:09.071043477 +0000 2024-09-17T09:23:14.262 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-17 09:20:09.071043477 +0000 2024-09-17T09:23:14.262 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-17 09:20:09.067043585 +0000 2024-09-17T09:23:14.262 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-17T09:23:14.325 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-17T09:23:14.325 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-17T09:23:14.326 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000240235 s, 2.1 MB/s 2024-09-17T09:23:14.327 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-17T09:23:14.384 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/nvme3n1 2024-09-17T09:23:14.440 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/nvme3n1 2024-09-17T09:23:14.441 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T09:23:14.441 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 1027 Links: 1 Device type: 103,6 2024-09-17T09:23:14.441 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T09:23:14.441 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T09:23:14.441 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-17 09:22:42.394890426 +0000 2024-09-17T09:23:14.441 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-17 09:20:08.938047076 +0000 2024-09-17T09:23:14.441 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-17 09:20:08.938047076 +0000 2024-09-17T09:23:14.441 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-17 09:20:08.933047212 +0000 2024-09-17T09:23:14.442 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-17T09:23:14.505 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-17T09:23:14.505 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-17T09:23:14.505 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000268977 s, 1.9 MB/s 2024-09-17T09:23:14.506 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-17T09:23:14.565 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/nvme2n1 2024-09-17T09:23:14.621 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/nvme2n1 2024-09-17T09:23:14.621 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T09:23:14.621 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 1015 Links: 1 Device type: 103,4 2024-09-17T09:23:14.621 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T09:23:14.621 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T09:23:14.621 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-17 09:22:42.383890724 +0000 2024-09-17T09:23:14.621 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-17 09:20:08.804050703 +0000 2024-09-17T09:23:14.621 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-17 09:20:08.804050703 +0000 2024-09-17T09:23:14.621 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-17 09:20:08.799050838 +0000 2024-09-17T09:23:14.622 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-17T09:23:14.685 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-17T09:23:14.685 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-17T09:23:14.685 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000202981 s, 2.5 MB/s 2024-09-17T09:23:14.687 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-17T09:23:14.743 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/nvme1n1 2024-09-17T09:23:14.799 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/nvme1n1 2024-09-17T09:23:14.800 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T09:23:14.800 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 1004 Links: 1 Device type: 103,2 2024-09-17T09:23:14.800 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T09:23:14.800 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T09:23:14.800 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-17 09:22:42.373890996 +0000 2024-09-17T09:23:14.800 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-17 09:20:08.668054384 +0000 2024-09-17T09:23:14.800 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-17 09:20:08.668054384 +0000 2024-09-17T09:23:14.800 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-17 09:20:08.663054519 +0000 2024-09-17T09:23:14.800 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-17T09:23:14.863 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-17T09:23:14.863 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-17T09:23:14.863 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000227479 s, 2.3 MB/s 2024-09-17T09:23:14.865 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-17T09:23:14.921 INFO:tasks.cephadm:Deploying osd.0 on smithi003 with /dev/nvme1n1... 2024-09-17T09:23:14.922 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme1n1 2024-09-17T09:23:15.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:14 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:15.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:14 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:15.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:14 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:15.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:14 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:15.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:14 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:15.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:14 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:15.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:14 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T09:23:15.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:14 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T09:23:15.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:14 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T09:23:15.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:14 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:15.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:14 smithi183 ceph-mon[30372]: Deploying daemon mgr.z on smithi183 2024-09-17T09:23:15.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:14 smithi183 ceph-mon[30372]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:15.128 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:23:15.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:14 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:15.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:14 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:15.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:14 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:15.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:14 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:15.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:14 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:15.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:14 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:15.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:14 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T09:23:15.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:14 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T09:23:15.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:14 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T09:23:15.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:14 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:15.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:14 smithi003 ceph-mon[25908]: Deploying daemon mgr.z on smithi183 2024-09-17T09:23:15.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:14 smithi003 ceph-mon[25908]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:15.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:14 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:15.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:14 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:15.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:14 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:15.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:14 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:15.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:14 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:15.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:14 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:15.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:14 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T09:23:15.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:14 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T09:23:15.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:14 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T09:23:15.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:14 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:15.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:14 smithi077 ceph-mon[30659]: Deploying daemon mgr.z on smithi183 2024-09-17T09:23:15.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:14 smithi077 ceph-mon[30659]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:16.242 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:16 smithi183 podman[32409]: 2024-09-17 09:23:16.029276917 +0000 UTC m=+0.015263714 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T09:23:16.499 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:16 smithi003 ceph-mon[25908]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:16.698 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:16 smithi183 ceph-mon[30372]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:16.698 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:16 smithi183 podman[32409]: 2024-09-17 09:23:16.363459668 +0000 UTC m=+0.349446464 container init 18462c45a5cb44f30e6ef9ddfba886de59914495e8279ba42cfd260cd3999667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T09:23:16.698 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:16 smithi183 podman[32409]: 2024-09-17 09:23:16.36841675 +0000 UTC m=+0.354403549 container start 18462c45a5cb44f30e6ef9ddfba886de59914495e8279ba42cfd260cd3999667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-17T09:23:16.698 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:16 smithi183 ceph-mgr[32447]: -- 172.21.15.183:0/2606438112 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561619e24d00 con 0x561619df9400 2024-09-17T09:23:16.698 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:16 smithi183 bash[32409]: 18462c45a5cb44f30e6ef9ddfba886de59914495e8279ba42cfd260cd3999667 2024-09-17T09:23:16.698 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:16 smithi183 systemd[1]: Started Ceph mgr.z for 08f126da-74d6-11ef-bceb-c7b262605968. 2024-09-17T09:23:16.698 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:16 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:16.486+0000 7f3ce2a1c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T09:23:16.698 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:16 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:16.546+0000 7f3ce2a1c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T09:23:16.699 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:16 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:16.695+0000 7f3ce2a1c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T09:23:16.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:16 smithi077 ceph-mon[30659]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:16.964 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping: /dev/nvme1n1 2024-09-17T09:23:16.964 INFO:teuthology.orchestra.run.smithi003.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-17T09:23:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2024-09-17T09:23:16.983 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10+0 records in 2024-09-17T09:23:16.983 INFO:teuthology.orchestra.run.smithi003.stderr:10+0 records out 2024-09-17T09:23:16.983 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123987 s, 846 MB/s 2024-09-17T09:23:16.983 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping successful for: 2024-09-17T09:23:17.042 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:16 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:16.761+0000 7f3ce2a1c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T09:23:17.042 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:16 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:16.903+0000 7f3ce2a1c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:23:17.042 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:17 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:17.039+0000 7f3ce2a1c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T09:23:17.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:17 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:17.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:17 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:17.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:17 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:17.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:17 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T09:23:17.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:17 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T09:23:17.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:17 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T09:23:17.546 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:17 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:17.546 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:17 smithi003 ceph-mon[25908]: Deploying daemon mgr.x on smithi077 2024-09-17T09:23:17.570 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi003:/dev/nvme1n1 2024-09-17T09:23:17.782 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:23:17.814 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:17 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:17.347+0000 7f3ce2a1c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T09:23:17.814 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:17 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:17.407+0000 7f3ce2a1c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T09:23:17.814 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:17 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:17.590+0000 7f3ce2a1c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T09:23:17.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:17 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:17.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:17 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:17.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:17 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:17.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:17 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T09:23:17.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:17 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T09:23:17.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:17 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T09:23:17.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:17 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:17.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:17 smithi183 ceph-mon[30372]: Deploying daemon mgr.x on smithi077 2024-09-17T09:23:17.838 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:17 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:17.838 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:17 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:17.838 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:17 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:17.838 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:17 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T09:23:17.838 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:17 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T09:23:17.838 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:17 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T09:23:17.838 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:17 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:17.838 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:17 smithi077 ceph-mon[30659]: Deploying daemon mgr.x on smithi077 2024-09-17T09:23:18.502 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:18 smithi077 podman[32073]: 2024-09-17 09:23:18.42364391 +0000 UTC m=+0.132653508 container create 0e9c55eeabe0a6660c0cd461469c4a60a82009731fedad038084c970278673dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-17T09:23:18.715 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:18 smithi183 ceph-mon[30372]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:18.716 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:18 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:18.453+0000 7f3ce2a1c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T09:23:18.716 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:18 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:18.518+0000 7f3ce2a1c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T09:23:18.788 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:18 smithi077 ceph-mon[30659]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:18.788 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:18 smithi077 podman[32073]: 2024-09-17 09:23:18.659552244 +0000 UTC m=+0.368561854 container init 0e9c55eeabe0a6660c0cd461469c4a60a82009731fedad038084c970278673dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-17T09:23:18.788 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:18 smithi077 podman[32073]: 2024-09-17 09:23:18.664604396 +0000 UTC m=+0.373614003 container start 0e9c55eeabe0a6660c0cd461469c4a60a82009731fedad038084c970278673dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-17T09:23:18.788 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:18 smithi077 ceph-mgr[32111]: -- 172.21.15.77:0/3160117415 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5617fd056d00 con 0x5617fd02b400 2024-09-17T09:23:18.788 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:18 smithi077 bash[32073]: 0e9c55eeabe0a6660c0cd461469c4a60a82009731fedad038084c970278673dd 2024-09-17T09:23:18.788 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:18 smithi077 systemd[1]: Started Ceph mgr.x for 08f126da-74d6-11ef-bceb-c7b262605968. 2024-09-17T09:23:18.850 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:18 smithi003 ceph-mon[25908]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:19.077 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:18 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:18.786+0000 7f0c71159040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T09:23:19.078 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:18 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:18.852+0000 7f0c71159040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T09:23:19.078 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:19 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:19.007+0000 7f0c71159040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T09:23:19.078 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:19 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:19.076+0000 7f0c71159040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T09:23:19.387 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:19 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:19.239+0000 7f0c71159040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:23:19.387 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:19 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:19.386+0000 7f0c71159040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T09:23:19.557 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:19 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:19.281+0000 7f3ce2a1c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T09:23:19.557 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:19 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:19.405+0000 7f3ce2a1c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T09:23:19.813 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:19 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:19.571+0000 7f3ce2a1c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:23:19.814 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:19 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:19.648+0000 7f3ce2a1c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T09:23:19.814 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:19 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:19.805+0000 7f3ce2a1c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T09:23:19.964 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:19 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:19.705+0000 7f0c71159040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T09:23:19.964 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:19 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:19.768+0000 7f0c71159040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T09:23:19.965 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:19 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:19.965 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:19 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:19.965 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:19 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:19.965 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:19 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:19.965 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:19 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:19.965 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:19 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:23:19.965 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:19 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:23:19.965 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:19 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:20.088 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:19 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:20.088 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:19 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:20.089 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:19 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:20.089 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:19 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:20.089 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:19 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:20.089 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:19 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:23:20.089 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:19 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:23:20.089 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:19 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:20.131 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:19 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:19.875+0000 7f3ce2a1c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T09:23:20.131 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:19 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:19.936+0000 7f3ce2a1c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T09:23:20.131 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:20 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:20.129+0000 7f3ce2a1c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T09:23:20.131 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:19 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:20.132 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:19 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:20.132 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:19 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:20.132 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:19 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:20.132 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:19 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:20.132 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:19 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:23:20.132 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:19 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:23:20.132 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:19 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:20.252 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:19 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:19.963+0000 7f0c71159040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T09:23:20.405 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:20 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:20.196+0000 7f3ce2a1c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T09:23:20.405 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:20 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:20.263+0000 7f3ce2a1c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T09:23:20.405 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:20 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:20.345+0000 7f3ce2a1c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T09:23:20.405 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:20 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:20.404+0000 7f3ce2a1c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T09:23:20.813 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:20 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:20.462+0000 7f3ce2a1c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T09:23:21.205 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:20 smithi003 ceph-mon[25908]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:23:21.206 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:20 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:21.206 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:20 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:21.206 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:20 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:21.206 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:20 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:21.206 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:20 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:21.206 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:20 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:21.206 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:20 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:21.206 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:20 smithi003 ceph-mon[25908]: Reconfiguring mgr.y (unknown last config time)... 2024-09-17T09:23:21.206 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:20 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T09:23:21.206 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:20 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T09:23:21.206 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:20 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:21.206 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:20 smithi003 ceph-mon[25908]: Reconfiguring daemon mgr.y on smithi003 2024-09-17T09:23:21.207 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:20 smithi003 ceph-mon[25908]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:21.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:20 smithi183 ceph-mon[30372]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:23:21.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:20 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:21.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:20 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:21.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:20 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:21.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:20 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:21.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:20 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:21.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:20 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:21.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:20 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:21.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:20 smithi183 ceph-mon[30372]: Reconfiguring mgr.y (unknown last config time)... 2024-09-17T09:23:21.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:20 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T09:23:21.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:20 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T09:23:21.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:20 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:21.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:20 smithi183 ceph-mon[30372]: Reconfiguring daemon mgr.y on smithi003 2024-09-17T09:23:21.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:20 smithi183 ceph-mon[30372]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:21.253 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:20 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:20.830+0000 7f0c71159040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T09:23:21.253 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:20 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:20.895+0000 7f0c71159040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T09:23:21.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:20 smithi077 ceph-mon[30659]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:23:21.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:20 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:21.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:20 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:21.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:20 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:21.254 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:20 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:21.254 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:20 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:21.254 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:20 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:21.254 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:20 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:21.254 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:20 smithi077 ceph-mon[30659]: Reconfiguring mgr.y (unknown last config time)... 2024-09-17T09:23:21.254 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:20 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T09:23:21.254 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:20 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T09:23:21.254 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:20 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:21.254 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:20 smithi077 ceph-mon[30659]: Reconfiguring daemon mgr.y on smithi003 2024-09-17T09:23:21.254 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:20 smithi077 ceph-mon[30659]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:21.564 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:23:21 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:23:21.230+0000 7f3ce2a1c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T09:23:21.856 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:21 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:21.603+0000 7f0c71159040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T09:23:21.856 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:21 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:21.716+0000 7f0c71159040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T09:23:21.856 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:21 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:21.855+0000 7f0c71159040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:23:22.132 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:21 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:21.915+0000 7f0c71159040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T09:23:22.133 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:22 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:22.060+0000 7f0c71159040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T09:23:22.133 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:22 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:22.131+0000 7f0c71159040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T09:23:22.133 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:22 smithi077 ceph-mon[30659]: Standby manager daemon z started 2024-09-17T09:23:22.133 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:22 smithi077 ceph-mon[30659]: from='mgr.? 172.21.15.183:0/2287272532' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-17T09:23:22.133 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:22 smithi077 ceph-mon[30659]: from='mgr.? 172.21.15.183:0/2287272532' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T09:23:22.133 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:22 smithi077 ceph-mon[30659]: from='mgr.? 172.21.15.183:0/2287272532' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-17T09:23:22.133 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:22 smithi077 ceph-mon[30659]: from='mgr.? 172.21.15.183:0/2287272532' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T09:23:22.133 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:22 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:22.133 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:22 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:22.133 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:22 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:22.134 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:22 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:22.134 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:22 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:22.134 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:22 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:22.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:22 smithi003 ceph-mon[25908]: Standby manager daemon z started 2024-09-17T09:23:22.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:22 smithi003 ceph-mon[25908]: from='mgr.? 172.21.15.183:0/2287272532' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-17T09:23:22.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:22 smithi003 ceph-mon[25908]: from='mgr.? 172.21.15.183:0/2287272532' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T09:23:22.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:22 smithi003 ceph-mon[25908]: from='mgr.? 172.21.15.183:0/2287272532' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-17T09:23:22.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:22 smithi003 ceph-mon[25908]: from='mgr.? 172.21.15.183:0/2287272532' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T09:23:22.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:22 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:22.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:22 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:22.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:22 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:22.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:22 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:22.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:22 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:22.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:22 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:22.412 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:22 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:22.191+0000 7f0c71159040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T09:23:22.412 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:22 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:22.347+0000 7f0c71159040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T09:23:22.412 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:22 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:22.411+0000 7f0c71159040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T09:23:22.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:22 smithi183 ceph-mon[30372]: Standby manager daemon z started 2024-09-17T09:23:22.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:22 smithi183 ceph-mon[30372]: from='mgr.? 172.21.15.183:0/2287272532' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-17T09:23:22.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:22 smithi183 ceph-mon[30372]: from='mgr.? 172.21.15.183:0/2287272532' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T09:23:22.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:22 smithi183 ceph-mon[30372]: from='mgr.? 172.21.15.183:0/2287272532' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-17T09:23:22.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:22 smithi183 ceph-mon[30372]: from='mgr.? 172.21.15.183:0/2287272532' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T09:23:22.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:22 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:22.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:22 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:22.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:22 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:22.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:22 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:22.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:22 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:22.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:22 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:22.673 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:22 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:22.477+0000 7f0c71159040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T09:23:22.674 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:22 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:22.557+0000 7f0c71159040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T09:23:22.674 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:22 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:22.615+0000 7f0c71159040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T09:23:22.674 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:22 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:22.672+0000 7f0c71159040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T09:23:23.443 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:23 smithi077 ceph-mon[30659]: mgrmap e14: y(active, since 73s), standbys: z 2024-09-17T09:23:23.443 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:23 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-17T09:23:23.443 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:23 smithi077 ceph-mon[30659]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:23.443 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:23 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/1420908567' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9e011d8-c4e3-4c7c-ab85-95b39ce8d1b8"}]: dispatch 2024-09-17T09:23:23.443 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:23 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/1420908567' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9e011d8-c4e3-4c7c-ab85-95b39ce8d1b8"}]': finished 2024-09-17T09:23:23.444 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:23 smithi077 ceph-mon[30659]: osdmap e5: 1 total, 0 up, 1 in 2024-09-17T09:23:23.444 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:23 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:23.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:23 smithi003 ceph-mon[25908]: mgrmap e14: y(active, since 73s), standbys: z 2024-09-17T09:23:23.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:23 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-17T09:23:23.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:23 smithi003 ceph-mon[25908]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:23.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:23 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/1420908567' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9e011d8-c4e3-4c7c-ab85-95b39ce8d1b8"}]: dispatch 2024-09-17T09:23:23.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:23 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/1420908567' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9e011d8-c4e3-4c7c-ab85-95b39ce8d1b8"}]': finished 2024-09-17T09:23:23.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:23 smithi003 ceph-mon[25908]: osdmap e5: 1 total, 0 up, 1 in 2024-09-17T09:23:23.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:23 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:23.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:23 smithi183 ceph-mon[30372]: mgrmap e14: y(active, since 73s), standbys: z 2024-09-17T09:23:23.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:23 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-17T09:23:23.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:23 smithi183 ceph-mon[30372]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:23.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:23 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/1420908567' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9e011d8-c4e3-4c7c-ab85-95b39ce8d1b8"}]: dispatch 2024-09-17T09:23:23.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:23 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/1420908567' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9e011d8-c4e3-4c7c-ab85-95b39ce8d1b8"}]': finished 2024-09-17T09:23:23.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:23 smithi183 ceph-mon[30372]: osdmap e5: 1 total, 0 up, 1 in 2024-09-17T09:23:23.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:23 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:23.752 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:23:23 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:23:23.442+0000 7f0c71159040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T09:23:24.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:24 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:24.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:24 smithi183 ceph-mon[30372]: Standby manager daemon x started 2024-09-17T09:23:24.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:24 smithi183 ceph-mon[30372]: from='mgr.? 172.21.15.77:0/2635383089' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T09:23:24.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:24 smithi183 ceph-mon[30372]: from='mgr.? 172.21.15.77:0/2635383089' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T09:23:24.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:24 smithi183 ceph-mon[30372]: from='mgr.? 172.21.15.77:0/2635383089' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T09:23:24.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:24 smithi183 ceph-mon[30372]: from='mgr.? 172.21.15.77:0/2635383089' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T09:23:24.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:24 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:24.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:24 smithi077 ceph-mon[30659]: Standby manager daemon x started 2024-09-17T09:23:24.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:24 smithi077 ceph-mon[30659]: from='mgr.? 172.21.15.77:0/2635383089' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T09:23:24.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:24 smithi077 ceph-mon[30659]: from='mgr.? 172.21.15.77:0/2635383089' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T09:23:24.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:24 smithi077 ceph-mon[30659]: from='mgr.? 172.21.15.77:0/2635383089' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T09:23:24.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:24 smithi077 ceph-mon[30659]: from='mgr.? 172.21.15.77:0/2635383089' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T09:23:24.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:24 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:24.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:24 smithi003 ceph-mon[25908]: Standby manager daemon x started 2024-09-17T09:23:24.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:24 smithi003 ceph-mon[25908]: from='mgr.? 172.21.15.77:0/2635383089' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T09:23:24.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:24 smithi003 ceph-mon[25908]: from='mgr.? 172.21.15.77:0/2635383089' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T09:23:24.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:24 smithi003 ceph-mon[25908]: from='mgr.? 172.21.15.77:0/2635383089' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T09:23:24.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:24 smithi003 ceph-mon[25908]: from='mgr.? 172.21.15.77:0/2635383089' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T09:23:25.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:25 smithi183 ceph-mon[30372]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:25.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:25 smithi183 ceph-mon[30372]: mgrmap e15: y(active, since 76s), standbys: x, z 2024-09-17T09:23:25.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:25 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T09:23:25.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:25 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/1836651764' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:23:25.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:25 smithi077 ceph-mon[30659]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:25.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:25 smithi077 ceph-mon[30659]: mgrmap e15: y(active, since 76s), standbys: x, z 2024-09-17T09:23:25.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:25 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T09:23:25.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:25 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/1836651764' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:23:25.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:25 smithi003 ceph-mon[25908]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:25.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:25 smithi003 ceph-mon[25908]: mgrmap e15: y(active, since 76s), standbys: x, z 2024-09-17T09:23:25.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:25 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T09:23:25.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:25 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/1836651764' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:23:27.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:27 smithi003 ceph-mon[25908]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:27.563 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:27 smithi183 ceph-mon[30372]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:27.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:27 smithi077 ceph-mon[30659]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:28.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:28 smithi003 ceph-mon[25908]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:28.653 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:28 smithi077 ceph-mon[30659]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:28.813 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:28 smithi183 ceph-mon[30372]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:30.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:29 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-17T09:23:30.251 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:29 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:30.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:29 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-17T09:23:30.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:29 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:30.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:29 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-17T09:23:30.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:29 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:31.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:30 smithi077 ceph-mon[30659]: Deploying daemon osd.0 on smithi003 2024-09-17T09:23:31.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:30 smithi077 ceph-mon[30659]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:31.255 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:30 smithi003 ceph-mon[25908]: Deploying daemon osd.0 on smithi003 2024-09-17T09:23:31.255 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:30 smithi003 ceph-mon[25908]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:31.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:30 smithi183 ceph-mon[30372]: Deploying daemon osd.0 on smithi003 2024-09-17T09:23:31.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:30 smithi183 ceph-mon[30372]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:32.470 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:32 smithi003 ceph-mon[25908]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:32.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:32 smithi077 ceph-mon[30659]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:32.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:32 smithi183 ceph-mon[30372]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:34.503 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:34 smithi003 ceph-mon[25908]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:34.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:34 smithi077 ceph-mon[30659]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:34.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:34 smithi183 ceph-mon[30372]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:35.578 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:35 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:35.579 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:35 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:35.580 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:35 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:35.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:35 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:35.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:35 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:35.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:35 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:35.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:35 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:35.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:35 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:35.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:35 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:36.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:36 smithi003 ceph-mon[25908]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:36.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:36 smithi077 ceph-mon[30659]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:36.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:36 smithi183 ceph-mon[30372]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:38.471 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 0 on host 'smithi003' 2024-09-17T09:23:38.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:38 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:38.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:38 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:38.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:38 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:38.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:38 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:38.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:38 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:38.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:38 smithi003 ceph-mon[25908]: from='osd.0 [v2:172.21.15.3:6802/3828533275,v1:172.21.15.3:6803/3828533275]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T09:23:38.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:38 smithi003 ceph-mon[25908]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:38.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:38 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:38.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:38 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:38.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:38 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:38.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:38 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:38.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:38 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:38.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:38 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:38.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:38 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:38.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:38 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:38.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:38 smithi183 ceph-mon[30372]: from='osd.0 [v2:172.21.15.3:6802/3828533275,v1:172.21.15.3:6803/3828533275]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T09:23:38.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:38 smithi183 ceph-mon[30372]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:38.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:38 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:38.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:38 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:38.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:38 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:39.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:38 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:39.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:38 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:39.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:38 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:39.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:38 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:39.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:38 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:39.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:38 smithi077 ceph-mon[30659]: from='osd.0 [v2:172.21.15.3:6802/3828533275,v1:172.21.15.3:6803/3828533275]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T09:23:39.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:38 smithi077 ceph-mon[30659]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:39.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:38 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:39.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:38 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:39.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:38 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:39.071 DEBUG:teuthology.orchestra.run.smithi003:osd.0> sudo journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@osd.0.service 2024-09-17T09:23:39.114 INFO:tasks.cephadm:Deploying osd.1 on smithi077 with /dev/nvme1n1... 2024-09-17T09:23:39.115 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme1n1 2024-09-17T09:23:39.316 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:23:39.690 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:39 smithi077 ceph-mon[30659]: from='osd.0 [v2:172.21.15.3:6802/3828533275,v1:172.21.15.3:6803/3828533275]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-17T09:23:39.690 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:39 smithi077 ceph-mon[30659]: osdmap e6: 1 total, 0 up, 1 in 2024-09-17T09:23:39.691 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:39 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:39.691 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:39 smithi077 ceph-mon[30659]: from='osd.0 [v2:172.21.15.3:6802/3828533275,v1:172.21.15.3:6803/3828533275]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-09-17T09:23:40.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:39 smithi003 ceph-mon[25908]: from='osd.0 [v2:172.21.15.3:6802/3828533275,v1:172.21.15.3:6803/3828533275]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-17T09:23:40.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:39 smithi003 ceph-mon[25908]: osdmap e6: 1 total, 0 up, 1 in 2024-09-17T09:23:40.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:39 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:40.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:39 smithi003 ceph-mon[25908]: from='osd.0 [v2:172.21.15.3:6802/3828533275,v1:172.21.15.3:6803/3828533275]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-09-17T09:23:40.009 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:23:39 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-0[43429]: 2024-09-17T09:23:39.577+0000 7fece01a0640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T09:23:40.009 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:23:39 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-0[43429]: 2024-09-17T09:23:39.581+0000 7fecdbfc9640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T09:23:40.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:39 smithi183 ceph-mon[30372]: from='osd.0 [v2:172.21.15.3:6802/3828533275,v1:172.21.15.3:6803/3828533275]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-17T09:23:40.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:39 smithi183 ceph-mon[30372]: osdmap e6: 1 total, 0 up, 1 in 2024-09-17T09:23:40.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:39 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:40.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:39 smithi183 ceph-mon[30372]: from='osd.0 [v2:172.21.15.3:6802/3828533275,v1:172.21.15.3:6803/3828533275]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-09-17T09:23:40.821 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:40 smithi077 ceph-mon[30659]: from='osd.0 [v2:172.21.15.3:6802/3828533275,v1:172.21.15.3:6803/3828533275]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-17T09:23:40.821 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:40 smithi077 ceph-mon[30659]: osdmap e7: 1 total, 0 up, 1 in 2024-09-17T09:23:40.821 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:40 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:40.821 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:40 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:40.821 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:40 smithi077 ceph-mon[30659]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:41.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:40 smithi003 ceph-mon[25908]: from='osd.0 [v2:172.21.15.3:6802/3828533275,v1:172.21.15.3:6803/3828533275]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-17T09:23:41.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:40 smithi003 ceph-mon[25908]: osdmap e7: 1 total, 0 up, 1 in 2024-09-17T09:23:41.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:40 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:41.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:40 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:41.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:40 smithi003 ceph-mon[25908]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:41.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:40 smithi183 ceph-mon[30372]: from='osd.0 [v2:172.21.15.3:6802/3828533275,v1:172.21.15.3:6803/3828533275]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-17T09:23:41.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:40 smithi183 ceph-mon[30372]: osdmap e7: 1 total, 0 up, 1 in 2024-09-17T09:23:41.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:40 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:41.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:40 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:41.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:40 smithi183 ceph-mon[30372]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:41.124 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/nvme1n1 2024-09-17T09:23:41.124 INFO:teuthology.orchestra.run.smithi077.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-17T09:23:41.128 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2024-09-17T09:23:41.150 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-09-17T09:23:41.150 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-09-17T09:23:41.150 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0198194 s, 529 MB/s 2024-09-17T09:23:41.150 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-09-17T09:23:41.788 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi077:/dev/nvme1n1 2024-09-17T09:23:41.813 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:41 smithi077 ceph-mon[30659]: purged_snaps scrub starts 2024-09-17T09:23:41.813 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:41 smithi077 ceph-mon[30659]: purged_snaps scrub ok 2024-09-17T09:23:41.814 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:41 smithi077 ceph-mon[30659]: osd.0 [v2:172.21.15.3:6802/3828533275,v1:172.21.15.3:6803/3828533275] boot 2024-09-17T09:23:41.814 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:41 smithi077 ceph-mon[30659]: osdmap e8: 1 total, 1 up, 1 in 2024-09-17T09:23:41.814 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:41 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:41.979 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:23:42.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:41 smithi003 ceph-mon[25908]: purged_snaps scrub starts 2024-09-17T09:23:42.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:41 smithi003 ceph-mon[25908]: purged_snaps scrub ok 2024-09-17T09:23:42.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:41 smithi003 ceph-mon[25908]: osd.0 [v2:172.21.15.3:6802/3828533275,v1:172.21.15.3:6803/3828533275] boot 2024-09-17T09:23:42.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:41 smithi003 ceph-mon[25908]: osdmap e8: 1 total, 1 up, 1 in 2024-09-17T09:23:42.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:41 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:42.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:41 smithi183 ceph-mon[30372]: purged_snaps scrub starts 2024-09-17T09:23:42.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:41 smithi183 ceph-mon[30372]: purged_snaps scrub ok 2024-09-17T09:23:42.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:41 smithi183 ceph-mon[30372]: osd.0 [v2:172.21.15.3:6802/3828533275,v1:172.21.15.3:6803/3828533275] boot 2024-09-17T09:23:42.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:41 smithi183 ceph-mon[30372]: osdmap e8: 1 total, 1 up, 1 in 2024-09-17T09:23:42.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:41 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:43.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:42 smithi077 ceph-mon[30659]: osdmap e9: 1 total, 1 up, 1 in 2024-09-17T09:23:43.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:42 smithi077 ceph-mon[30659]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:43.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:42 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:43.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:42 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:43.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:42 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:43.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:42 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:43.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:42 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:43.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:42 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:43.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:42 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:43.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:42 smithi003 ceph-mon[25908]: osdmap e9: 1 total, 1 up, 1 in 2024-09-17T09:23:43.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:42 smithi003 ceph-mon[25908]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:43.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:42 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:43.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:42 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:43.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:42 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:43.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:42 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:43.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:42 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:43.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:42 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:43.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:42 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:43.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:42 smithi183 ceph-mon[30372]: osdmap e9: 1 total, 1 up, 1 in 2024-09-17T09:23:43.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:42 smithi183 ceph-mon[30372]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:43.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:42 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:43.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:42 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:43.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:42 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:43.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:42 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:43.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:42 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:43.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:42 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:43.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:42 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:43.605 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:43 smithi077 ceph-mon[30659]: Detected new or changed devices on smithi003 2024-09-17T09:23:43.605 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:43 smithi077 ceph-mon[30659]: Adjusting osd_memory_target on smithi003 to 17092M 2024-09-17T09:23:44.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:43 smithi003 ceph-mon[25908]: Detected new or changed devices on smithi003 2024-09-17T09:23:44.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:43 smithi003 ceph-mon[25908]: Adjusting osd_memory_target on smithi003 to 17092M 2024-09-17T09:23:44.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:43 smithi183 ceph-mon[30372]: Detected new or changed devices on smithi003 2024-09-17T09:23:44.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:43 smithi183 ceph-mon[30372]: Adjusting osd_memory_target on smithi003 to 17092M 2024-09-17T09:23:44.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:44 smithi077 ceph-mon[30659]: from='client.24128 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:23:44.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:44 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:23:44.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:44 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:23:44.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:44 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:44.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:44 smithi077 ceph-mon[30659]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:45.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:44 smithi003 ceph-mon[25908]: from='client.24128 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:23:45.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:44 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:23:45.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:44 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:23:45.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:44 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:45.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:44 smithi003 ceph-mon[25908]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:45.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:44 smithi183 ceph-mon[30372]: from='client.24128 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:23:45.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:44 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:23:45.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:44 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:23:45.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:44 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:45.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:44 smithi183 ceph-mon[30372]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:46.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:46 smithi183 ceph-mon[30372]: from='client.? 172.21.15.77:0/3250119893' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbfe87f9-1893-411d-a48e-85cfd24590f1"}]: dispatch 2024-09-17T09:23:46.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:46 smithi183 ceph-mon[30372]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbfe87f9-1893-411d-a48e-85cfd24590f1"}]: dispatch 2024-09-17T09:23:46.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:46 smithi183 ceph-mon[30372]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbfe87f9-1893-411d-a48e-85cfd24590f1"}]': finished 2024-09-17T09:23:46.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:46 smithi183 ceph-mon[30372]: osdmap e10: 2 total, 1 up, 2 in 2024-09-17T09:23:46.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:46 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:23:46.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:46 smithi077 ceph-mon[30659]: from='client.? 172.21.15.77:0/3250119893' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbfe87f9-1893-411d-a48e-85cfd24590f1"}]: dispatch 2024-09-17T09:23:46.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:46 smithi077 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbfe87f9-1893-411d-a48e-85cfd24590f1"}]: dispatch 2024-09-17T09:23:46.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:46 smithi077 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbfe87f9-1893-411d-a48e-85cfd24590f1"}]': finished 2024-09-17T09:23:46.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:46 smithi077 ceph-mon[30659]: osdmap e10: 2 total, 1 up, 2 in 2024-09-17T09:23:46.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:46 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:23:46.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:46 smithi003 ceph-mon[25908]: from='client.? 172.21.15.77:0/3250119893' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbfe87f9-1893-411d-a48e-85cfd24590f1"}]: dispatch 2024-09-17T09:23:46.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:46 smithi003 ceph-mon[25908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbfe87f9-1893-411d-a48e-85cfd24590f1"}]: dispatch 2024-09-17T09:23:46.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:46 smithi003 ceph-mon[25908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbfe87f9-1893-411d-a48e-85cfd24590f1"}]': finished 2024-09-17T09:23:46.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:46 smithi003 ceph-mon[25908]: osdmap e10: 2 total, 1 up, 2 in 2024-09-17T09:23:46.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:46 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:23:47.200 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:47 smithi077 ceph-mon[30659]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:47.313 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:47 smithi183 ceph-mon[30372]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:47.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:47 smithi003 ceph-mon[25908]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:48.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:48 smithi183 ceph-mon[30372]: from='client.? 172.21.15.77:0/2293895192' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:23:48.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:48 smithi077 ceph-mon[30659]: from='client.? 172.21.15.77:0/2293895192' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:23:48.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:48 smithi003 ceph-mon[25908]: from='client.? 172.21.15.77:0/2293895192' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:23:49.313 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:49 smithi183 ceph-mon[30372]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:49.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:49 smithi077 ceph-mon[30659]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:49.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:49 smithi003 ceph-mon[25908]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:50.610 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:50 smithi077 ceph-mon[30659]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:50.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:50 smithi003 ceph-mon[25908]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:50.813 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:50 smithi183 ceph-mon[30372]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:52.410 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:52 smithi077 ceph-mon[30659]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:52.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:52 smithi003 ceph-mon[25908]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:52.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:52 smithi183 ceph-mon[30372]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:53.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:53 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-17T09:23:53.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:53 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:53.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:53 smithi077 ceph-mon[30659]: Deploying daemon osd.1 on smithi077 2024-09-17T09:23:53.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:53 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-17T09:23:53.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:53 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:53.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:53 smithi003 ceph-mon[25908]: Deploying daemon osd.1 on smithi077 2024-09-17T09:23:53.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:53 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-17T09:23:53.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:53 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:53.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:53 smithi183 ceph-mon[30372]: Deploying daemon osd.1 on smithi077 2024-09-17T09:23:54.605 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:54 smithi077 ceph-mon[30659]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:54.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:54 smithi003 ceph-mon[25908]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:54.813 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:54 smithi183 ceph-mon[30372]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:56.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:56 smithi077 ceph-mon[30659]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:56.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:56 smithi003 ceph-mon[25908]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:56.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:56 smithi183 ceph-mon[30372]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:57.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:57 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:57.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:57 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:57.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:57 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:57.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:57 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:57.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:57 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:57.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:57 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:57.813 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:57 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:57.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:57 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:57.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:57 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:23:58.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:23:58 smithi077 ceph-mon[30659]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:58.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:23:58 smithi003 ceph-mon[25908]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:58.817 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:23:58 smithi183 ceph-mon[30372]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:24:00.474 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:00 smithi077 ceph-mon[30659]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:24:00.507 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 1 on host 'smithi077' 2024-09-17T09:24:00.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:00 smithi003 ceph-mon[25908]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:24:00.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:00 smithi183 ceph-mon[30372]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:24:01.089 DEBUG:teuthology.orchestra.run.smithi077:osd.1> sudo journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@osd.1.service 2024-09-17T09:24:01.131 INFO:tasks.cephadm:Deploying osd.2 on smithi183 with /dev/nvme1n1... 2024-09-17T09:24:01.131 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme1n1 2024-09-17T09:24:01.334 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T09:24:01.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:01.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:01.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:01 smithi077 ceph-mon[30659]: from='osd.1 [v2:172.21.15.77:6800/4104131602,v1:172.21.15.77:6801/4104131602]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T09:24:01.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:01 smithi077 ceph-mon[30659]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T09:24:01.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:01.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:01.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:01 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:01.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:01 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:01.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:01 smithi003 ceph-mon[25908]: from='osd.1 [v2:172.21.15.77:6800/4104131602,v1:172.21.15.77:6801/4104131602]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T09:24:01.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:01 smithi003 ceph-mon[25908]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T09:24:01.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:01 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:01.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:01 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:01.791 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:01 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:01.792 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:01 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:01.792 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:01 smithi183 ceph-mon[30372]: from='osd.1 [v2:172.21.15.77:6800/4104131602,v1:172.21.15.77:6801/4104131602]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T09:24:01.792 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:01 smithi183 ceph-mon[30372]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T09:24:01.792 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:01 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:01.792 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:01 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:02.697 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:02 smithi183 ceph-mon[30372]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-17T09:24:02.697 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:02 smithi183 ceph-mon[30372]: osdmap e11: 2 total, 1 up, 2 in 2024-09-17T09:24:02.697 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:02 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:24:02.698 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:02 smithi183 ceph-mon[30372]: from='osd.1 [v2:172.21.15.77:6800/4104131602,v1:172.21.15.77:6801/4104131602]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-17T09:24:02.698 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:02 smithi183 ceph-mon[30372]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-17T09:24:02.698 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:02 smithi183 ceph-mon[30372]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:24:03.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:02 smithi077 ceph-mon[30659]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-17T09:24:03.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:02 smithi077 ceph-mon[30659]: osdmap e11: 2 total, 1 up, 2 in 2024-09-17T09:24:03.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:02 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:24:03.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:02 smithi077 ceph-mon[30659]: from='osd.1 [v2:172.21.15.77:6800/4104131602,v1:172.21.15.77:6801/4104131602]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-17T09:24:03.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:02 smithi077 ceph-mon[30659]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-17T09:24:03.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:02 smithi077 ceph-mon[30659]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:24:03.003 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:24:02 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-1[35061]: 2024-09-17T09:24:02.513+0000 7fe5eb83a640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T09:24:03.003 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:24:02 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-1[35061]: 2024-09-17T09:24:02.515+0000 7fe5e7663640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T09:24:03.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:02 smithi003 ceph-mon[25908]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-17T09:24:03.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:02 smithi003 ceph-mon[25908]: osdmap e11: 2 total, 1 up, 2 in 2024-09-17T09:24:03.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:02 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:24:03.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:02 smithi003 ceph-mon[25908]: from='osd.1 [v2:172.21.15.77:6800/4104131602,v1:172.21.15.77:6801/4104131602]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-17T09:24:03.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:02 smithi003 ceph-mon[25908]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-17T09:24:03.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:02 smithi003 ceph-mon[25908]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:24:03.136 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/nvme1n1 2024-09-17T09:24:03.136 INFO:teuthology.orchestra.run.smithi183.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-17T09:24:03.140 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2024-09-17T09:24:03.155 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-09-17T09:24:03.155 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-09-17T09:24:03.155 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129066 s, 812 MB/s 2024-09-17T09:24:03.155 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-09-17T09:24:03.713 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:03 smithi077 ceph-mon[30659]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-17T09:24:03.713 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:03 smithi077 ceph-mon[30659]: osdmap e12: 2 total, 1 up, 2 in 2024-09-17T09:24:03.713 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:03 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:24:03.713 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:03 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:24:03.760 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:03 smithi183 ceph-mon[30372]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-17T09:24:03.760 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:03 smithi183 ceph-mon[30372]: osdmap e12: 2 total, 1 up, 2 in 2024-09-17T09:24:03.760 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:03 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:24:03.760 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:03 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:24:03.781 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi183:/dev/nvme1n1 2024-09-17T09:24:03.971 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T09:24:04.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:03 smithi003 ceph-mon[25908]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-17T09:24:04.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:03 smithi003 ceph-mon[25908]: osdmap e12: 2 total, 1 up, 2 in 2024-09-17T09:24:04.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:03 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:24:04.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:03 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:24:04.757 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:04 smithi183 ceph-mon[30372]: purged_snaps scrub starts 2024-09-17T09:24:04.757 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:04 smithi183 ceph-mon[30372]: purged_snaps scrub ok 2024-09-17T09:24:04.757 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:04 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:24:04.757 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:04 smithi183 ceph-mon[30372]: osd.1 [v2:172.21.15.77:6800/4104131602,v1:172.21.15.77:6801/4104131602] boot 2024-09-17T09:24:04.758 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:04 smithi183 ceph-mon[30372]: osdmap e13: 2 total, 2 up, 2 in 2024-09-17T09:24:04.758 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:04 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:24:04.758 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:04 smithi183 ceph-mon[30372]: Detected new or changed devices on smithi077 2024-09-17T09:24:04.758 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:04 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:04.758 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:04 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:04.758 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:04 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:24:04.758 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:04 smithi183 ceph-mon[30372]: Adjusting osd_memory_target on smithi077 to 17088M 2024-09-17T09:24:04.758 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:04 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:04.758 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:04 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:04.758 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:04 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:04.758 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:04 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:04.758 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:04 smithi183 ceph-mon[30372]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:24:04.758 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:04 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:04.759 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:04 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:04.759 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:04 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:04.759 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:04 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:05.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:04 smithi077 ceph-mon[30659]: purged_snaps scrub starts 2024-09-17T09:24:05.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:04 smithi077 ceph-mon[30659]: purged_snaps scrub ok 2024-09-17T09:24:05.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:04 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:24:05.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:04 smithi077 ceph-mon[30659]: osd.1 [v2:172.21.15.77:6800/4104131602,v1:172.21.15.77:6801/4104131602] boot 2024-09-17T09:24:05.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:04 smithi077 ceph-mon[30659]: osdmap e13: 2 total, 2 up, 2 in 2024-09-17T09:24:05.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:04 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:04 smithi077 ceph-mon[30659]: Detected new or changed devices on smithi077 2024-09-17T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:04 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:04 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:04 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:04 smithi077 ceph-mon[30659]: Adjusting osd_memory_target on smithi077 to 17088M 2024-09-17T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:04 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:04 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:04 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:04 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:04 smithi077 ceph-mon[30659]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:04 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:04 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:05.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:04 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:05.004 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:04 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:05.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:04 smithi003 ceph-mon[25908]: purged_snaps scrub starts 2024-09-17T09:24:05.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:04 smithi003 ceph-mon[25908]: purged_snaps scrub ok 2024-09-17T09:24:05.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:04 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:24:05.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:04 smithi003 ceph-mon[25908]: osd.1 [v2:172.21.15.77:6800/4104131602,v1:172.21.15.77:6801/4104131602] boot 2024-09-17T09:24:05.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:04 smithi003 ceph-mon[25908]: osdmap e13: 2 total, 2 up, 2 in 2024-09-17T09:24:05.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:04 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:24:05.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:04 smithi003 ceph-mon[25908]: Detected new or changed devices on smithi077 2024-09-17T09:24:05.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:04 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:05.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:04 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:05.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:04 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:24:05.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:04 smithi003 ceph-mon[25908]: Adjusting osd_memory_target on smithi077 to 17088M 2024-09-17T09:24:05.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:04 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:05.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:04 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:05.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:04 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:05.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:04 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:05.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:04 smithi003 ceph-mon[25908]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:24:05.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:04 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:05.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:04 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:05.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:04 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:05.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:04 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:05.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:05 smithi183 ceph-mon[30372]: osdmap e14: 2 total, 2 up, 2 in 2024-09-17T09:24:06.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:05 smithi077 ceph-mon[30659]: osdmap e14: 2 total, 2 up, 2 in 2024-09-17T09:24:06.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:05 smithi003 ceph-mon[25908]: osdmap e14: 2 total, 2 up, 2 in 2024-09-17T09:24:06.742 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:06 smithi183 ceph-mon[30372]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:06.742 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:06 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:24:06.742 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:06 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:24:06.742 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:06 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:06.742 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:06 smithi183 ceph-mon[30372]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:07.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:06 smithi077 ceph-mon[30659]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:07.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:06 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:24:07.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:06 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:24:07.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:06 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:07.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:06 smithi077 ceph-mon[30659]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:07.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:06 smithi003 ceph-mon[25908]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:07.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:06 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:24:07.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:06 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:24:07.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:06 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:07.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:06 smithi003 ceph-mon[25908]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:08.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:08 smithi183 ceph-mon[30372]: from='client.? 172.21.15.183:0/1143381888' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474107a4-c9ef-4cbb-8e7c-3decc0835717"}]: dispatch 2024-09-17T09:24:08.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:08 smithi183 ceph-mon[30372]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474107a4-c9ef-4cbb-8e7c-3decc0835717"}]: dispatch 2024-09-17T09:24:08.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:08 smithi183 ceph-mon[30372]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "474107a4-c9ef-4cbb-8e7c-3decc0835717"}]': finished 2024-09-17T09:24:08.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:08 smithi183 ceph-mon[30372]: osdmap e15: 3 total, 2 up, 3 in 2024-09-17T09:24:08.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:08 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:24:08.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:08 smithi077 ceph-mon[30659]: from='client.? 172.21.15.183:0/1143381888' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474107a4-c9ef-4cbb-8e7c-3decc0835717"}]: dispatch 2024-09-17T09:24:08.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:08 smithi077 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474107a4-c9ef-4cbb-8e7c-3decc0835717"}]: dispatch 2024-09-17T09:24:08.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:08 smithi077 ceph-mon[30659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "474107a4-c9ef-4cbb-8e7c-3decc0835717"}]': finished 2024-09-17T09:24:08.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:08 smithi077 ceph-mon[30659]: osdmap e15: 3 total, 2 up, 3 in 2024-09-17T09:24:08.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:08 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:24:08.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:08 smithi003 ceph-mon[25908]: from='client.? 172.21.15.183:0/1143381888' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474107a4-c9ef-4cbb-8e7c-3decc0835717"}]: dispatch 2024-09-17T09:24:08.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:08 smithi003 ceph-mon[25908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474107a4-c9ef-4cbb-8e7c-3decc0835717"}]: dispatch 2024-09-17T09:24:08.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:08 smithi003 ceph-mon[25908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "474107a4-c9ef-4cbb-8e7c-3decc0835717"}]': finished 2024-09-17T09:24:08.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:08 smithi003 ceph-mon[25908]: osdmap e15: 3 total, 2 up, 3 in 2024-09-17T09:24:08.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:08 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:24:09.195 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:09 smithi183 ceph-mon[30372]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:09.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:09 smithi077 ceph-mon[30659]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:09.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:09 smithi003 ceph-mon[25908]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:10.313 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:10 smithi183 ceph-mon[30372]: from='client.? 172.21.15.183:0/222570697' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:24:10.373 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:10 smithi077 ceph-mon[30659]: from='client.? 172.21.15.183:0/222570697' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:24:10.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:10 smithi003 ceph-mon[25908]: from='client.? 172.21.15.183:0/222570697' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:24:11.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:11 smithi183 ceph-mon[30372]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:11.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:11 smithi077 ceph-mon[30659]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:11.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:11 smithi003 ceph-mon[25908]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:12.563 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:12 smithi183 ceph-mon[30372]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:12.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:12 smithi077 ceph-mon[30659]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:12.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:12 smithi003 ceph-mon[25908]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:14.365 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:14 smithi183 ceph-mon[30372]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:14.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:14 smithi077 ceph-mon[30659]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:14.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:14 smithi003 ceph-mon[25908]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:15.620 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:15 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-17T09:24:15.621 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:15 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:15.621 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:15 smithi183 ceph-mon[30372]: Deploying daemon osd.2 on smithi183 2024-09-17T09:24:15.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:15 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-17T09:24:15.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:15 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:15.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:15 smithi077 ceph-mon[30659]: Deploying daemon osd.2 on smithi183 2024-09-17T09:24:15.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:15 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-17T09:24:15.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:15 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:15.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:15 smithi003 ceph-mon[25908]: Deploying daemon osd.2 on smithi183 2024-09-17T09:24:16.485 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:16 smithi183 ceph-mon[30372]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:16.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:16 smithi003 ceph-mon[25908]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:16.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:16 smithi077 ceph-mon[30659]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:18.695 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:18 smithi183 ceph-mon[30372]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:18.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:18 smithi077 ceph-mon[30659]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:18.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:18 smithi003 ceph-mon[25908]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:19.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:19 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:19.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:19 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:19.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:19 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:19.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:19 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:19.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:19 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:19.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:19 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:19.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:19 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:19.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:19 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:19.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:19 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:20.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:20 smithi183 ceph-mon[30372]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:20.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:20 smithi077 ceph-mon[30659]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:20.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:20 smithi003 ceph-mon[25908]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:22.354 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 2 on host 'smithi183' 2024-09-17T09:24:22.691 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:22 smithi183 ceph-mon[30372]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:22.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:22 smithi077 ceph-mon[30659]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:22.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:22 smithi003 ceph-mon[25908]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:22.978 DEBUG:teuthology.orchestra.run.smithi183:osd.2> sudo journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@osd.2.service 2024-09-17T09:24:22.980 INFO:tasks.cephadm:Waiting for 3 OSDs to come up... 2024-09-17T09:24:22.981 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-17T09:24:23.190 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:24:23.543 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:23 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:23.544 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:23 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:23.544 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:23 smithi183 ceph-mon[30372]: from='osd.2 [v2:172.21.15.183:6800/711224978,v1:172.21.15.183:6801/711224978]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T09:24:23.544 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:23 smithi183 ceph-mon[30372]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T09:24:23.544 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:23 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:23.544 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:23 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:23.572 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:23 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:23.572 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:23 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:23.572 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:23 smithi003 ceph-mon[25908]: from='osd.2 [v2:172.21.15.183:6800/711224978,v1:172.21.15.183:6801/711224978]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T09:24:23.572 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:23 smithi003 ceph-mon[25908]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T09:24:23.572 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:23 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:23.572 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:23 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:23.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:23 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:23.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:23 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:23.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:23 smithi077 ceph-mon[30659]: from='osd.2 [v2:172.21.15.183:6800/711224978,v1:172.21.15.183:6801/711224978]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T09:24:23.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:23 smithi077 ceph-mon[30659]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T09:24:23.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:23 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:23.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:23 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:24.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:24 smithi183 ceph-mon[30372]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-17T09:24:24.464 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:24 smithi183 ceph-mon[30372]: osdmap e16: 3 total, 2 up, 3 in 2024-09-17T09:24:24.464 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:24 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:24:24.464 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:24 smithi183 ceph-mon[30372]: from='osd.2 [v2:172.21.15.183:6800/711224978,v1:172.21.15.183:6801/711224978]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-17T09:24:24.464 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:24 smithi183 ceph-mon[30372]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-17T09:24:24.464 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:24 smithi183 ceph-mon[30372]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:24.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:24 smithi003 ceph-mon[25908]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-17T09:24:24.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:24 smithi003 ceph-mon[25908]: osdmap e16: 3 total, 2 up, 3 in 2024-09-17T09:24:24.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:24 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:24:24.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:24 smithi003 ceph-mon[25908]: from='osd.2 [v2:172.21.15.183:6800/711224978,v1:172.21.15.183:6801/711224978]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-17T09:24:24.654 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:24 smithi003 ceph-mon[25908]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-17T09:24:24.654 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:24 smithi003 ceph-mon[25908]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:24.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:24 smithi077 ceph-mon[30659]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-17T09:24:24.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:24 smithi077 ceph-mon[30659]: osdmap e16: 3 total, 2 up, 3 in 2024-09-17T09:24:24.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:24 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:24:24.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:24 smithi077 ceph-mon[30659]: from='osd.2 [v2:172.21.15.183:6800/711224978,v1:172.21.15.183:6801/711224978]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-17T09:24:24.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:24 smithi077 ceph-mon[30659]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-17T09:24:24.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:24 smithi077 ceph-mon[30659]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:25.105 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:24:25.314 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:24:24 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-2[35479]: 2024-09-17T09:24:24.820+0000 7f4bbe01e640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T09:24:25.314 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:24:24 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-2[35479]: 2024-09-17T09:24:24.823+0000 7f4bb5e3f640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T09:24:25.678 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:25 smithi183 ceph-mon[30372]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-17T09:24:25.678 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:25 smithi183 ceph-mon[30372]: osdmap e17: 3 total, 2 up, 3 in 2024-09-17T09:24:25.679 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:25 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:24:25.679 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:25 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:24:25.679 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:25 smithi183 ceph-mon[30372]: from='osd.2 ' entity='osd.2' 2024-09-17T09:24:25.679 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:25 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/2819428404' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-17T09:24:25.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:25 smithi003 ceph-mon[25908]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-17T09:24:25.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:25 smithi003 ceph-mon[25908]: osdmap e17: 3 total, 2 up, 3 in 2024-09-17T09:24:25.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:25 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:24:25.690 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:25 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:24:25.690 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:25 smithi003 ceph-mon[25908]: from='osd.2 ' entity='osd.2' 2024-09-17T09:24:25.690 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:25 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/2819428404' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-17T09:24:25.713 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"num_osds":3,"num_up_osds":2,"osd_up_since":1726565043,"num_in_osds":3,"osd_in_since":1726565047,"num_remapped_pgs":0} 2024-09-17T09:24:25.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:25 smithi077 ceph-mon[30659]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-17T09:24:25.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:25 smithi077 ceph-mon[30659]: osdmap e17: 3 total, 2 up, 3 in 2024-09-17T09:24:25.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:25 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:24:25.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:25 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:24:25.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:25 smithi077 ceph-mon[30659]: from='osd.2 ' entity='osd.2' 2024-09-17T09:24:25.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:25 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/2819428404' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-17T09:24:26.714 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-17T09:24:26.911 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:24:27.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:26 smithi077 ceph-mon[30659]: purged_snaps scrub starts 2024-09-17T09:24:27.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:26 smithi077 ceph-mon[30659]: purged_snaps scrub ok 2024-09-17T09:24:27.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:26 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:24:27.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:26 smithi077 ceph-mon[30659]: osd.2 [v2:172.21.15.183:6800/711224978,v1:172.21.15.183:6801/711224978] boot 2024-09-17T09:24:27.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:26 smithi077 ceph-mon[30659]: osdmap e18: 3 total, 3 up, 3 in 2024-09-17T09:24:27.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:26 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:24:27.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:26 smithi077 ceph-mon[30659]: Detected new or changed devices on smithi183 2024-09-17T09:24:27.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:26 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:27.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:26 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:27.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:26 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:24:27.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:26 smithi077 ceph-mon[30659]: Adjusting osd_memory_target on smithi183 to 17088M 2024-09-17T09:24:27.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:26 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:27.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:26 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:27.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:26 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:27.004 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:26 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:27.004 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:26 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:27.004 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:26 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:27.004 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:26 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:27.004 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:26 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:27.004 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:26 smithi077 ceph-mon[30659]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:27.004 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:26 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-17T09:24:27.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:26 smithi003 ceph-mon[25908]: purged_snaps scrub starts 2024-09-17T09:24:27.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:26 smithi003 ceph-mon[25908]: purged_snaps scrub ok 2024-09-17T09:24:27.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:26 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:24:27.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:26 smithi003 ceph-mon[25908]: osd.2 [v2:172.21.15.183:6800/711224978,v1:172.21.15.183:6801/711224978] boot 2024-09-17T09:24:27.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:26 smithi003 ceph-mon[25908]: osdmap e18: 3 total, 3 up, 3 in 2024-09-17T09:24:27.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:26 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:24:27.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:26 smithi003 ceph-mon[25908]: Detected new or changed devices on smithi183 2024-09-17T09:24:27.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:26 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:27.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:26 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:27.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:26 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:24:27.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:26 smithi003 ceph-mon[25908]: Adjusting osd_memory_target on smithi183 to 17088M 2024-09-17T09:24:27.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:26 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:27.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:26 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:27.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:26 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:27.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:26 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:27.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:26 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:27.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:26 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:27.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:26 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:27.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:26 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:27.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:26 smithi003 ceph-mon[25908]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:27.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:26 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-17T09:24:27.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:26 smithi183 ceph-mon[30372]: purged_snaps scrub starts 2024-09-17T09:24:27.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:26 smithi183 ceph-mon[30372]: purged_snaps scrub ok 2024-09-17T09:24:27.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:26 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:24:27.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:26 smithi183 ceph-mon[30372]: osd.2 [v2:172.21.15.183:6800/711224978,v1:172.21.15.183:6801/711224978] boot 2024-09-17T09:24:27.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:26 smithi183 ceph-mon[30372]: osdmap e18: 3 total, 3 up, 3 in 2024-09-17T09:24:27.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:26 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:24:27.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:26 smithi183 ceph-mon[30372]: Detected new or changed devices on smithi183 2024-09-17T09:24:27.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:26 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:27.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:26 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:27.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:26 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:24:27.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:26 smithi183 ceph-mon[30372]: Adjusting osd_memory_target on smithi183 to 17088M 2024-09-17T09:24:27.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:26 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:27.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:26 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:27.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:26 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:27.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:26 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:27.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:26 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:27.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:26 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:27.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:26 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:27.066 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:26 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:27.066 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:26 smithi183 ceph-mon[30372]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:27.066 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:26 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-17T09:24:28.432 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:28 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-17T09:24:28.432 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:28 smithi003 ceph-mon[25908]: osdmap e19: 3 total, 3 up, 3 in 2024-09-17T09:24:28.432 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:28 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T09:24:28.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:28 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-17T09:24:28.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:28 smithi077 ceph-mon[30659]: osdmap e19: 3 total, 3 up, 3 in 2024-09-17T09:24:28.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:28 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T09:24:28.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:28 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-17T09:24:28.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:28 smithi183 ceph-mon[30372]: osdmap e19: 3 total, 3 up, 3 in 2024-09-17T09:24:28.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:28 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T09:24:28.831 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:24:29.008 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:24:28 smithi003 sudo[49574]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1 2024-09-17T09:24:29.008 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:24:28 smithi003 sudo[49574]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:29.008 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:24:28 smithi003 sudo[49574]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:29.008 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:24:28 smithi003 sudo[49574]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:29.410 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:24:29 smithi003 sudo[49598]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme1n1 2024-09-17T09:24:29.410 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:24:29 smithi003 sudo[49598]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:29.410 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:24:29 smithi003 sudo[49598]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:29.410 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:24:29 smithi003 sudo[49598]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:29.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:29 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-17T09:24:29.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:29 smithi003 ceph-mon[25908]: osdmap e20: 3 total, 3 up, 3 in 2024-09-17T09:24:29.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:29 smithi003 ceph-mon[25908]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:29.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:29 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/346953777' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-17T09:24:29.431 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"num_osds":3,"num_up_osds":3,"osd_up_since":1726565065,"num_in_osds":3,"osd_in_since":1726565047,"num_remapped_pgs":0} 2024-09-17T09:24:29.431 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-17T09:24:29.440 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:29 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-17T09:24:29.441 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:29 smithi077 ceph-mon[30659]: osdmap e20: 3 total, 3 up, 3 in 2024-09-17T09:24:29.441 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:29 smithi077 ceph-mon[30659]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:29.441 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:29 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/346953777' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-17T09:24:29.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:29 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-17T09:24:29.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:29 smithi183 ceph-mon[30372]: osdmap e20: 3 total, 3 up, 3 in 2024-09-17T09:24:29.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:29 smithi183 ceph-mon[30372]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:29.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:29 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/346953777' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-17T09:24:29.626 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:24:29.699 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:24:29 smithi077 sudo[39932]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1 2024-09-17T09:24:29.699 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:24:29 smithi077 sudo[39932]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:29.699 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:24:29 smithi077 sudo[39932]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:29.700 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:24:29 smithi077 sudo[39932]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:30.002 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:24:29 smithi077 sudo[39936]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme1n1 2024-09-17T09:24:30.002 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:24:29 smithi077 sudo[39936]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:30.002 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:24:29 smithi077 sudo[39936]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:30.002 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:24:29 smithi077 sudo[39936]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:30.225 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:30 smithi183 ceph-mon[30372]: mgrmap e16: y(active, since 2m), standbys: x, z 2024-09-17T09:24:30.226 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:30 smithi183 ceph-mon[30372]: osdmap e21: 3 total, 3 up, 3 in 2024-09-17T09:24:30.226 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:30 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:30.226 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:30 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:30.226 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:24:29 smithi183 sudo[40286]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1 2024-09-17T09:24:30.226 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:24:29 smithi183 sudo[40286]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:30.226 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:24:29 smithi183 sudo[40286]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:30.226 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:24:29 smithi183 sudo[40286]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:30.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:30 smithi077 ceph-mon[30659]: mgrmap e16: y(active, since 2m), standbys: x, z 2024-09-17T09:24:30.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:30 smithi077 ceph-mon[30659]: osdmap e21: 3 total, 3 up, 3 in 2024-09-17T09:24:30.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:30 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:30.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:30 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:30.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:30 smithi003 ceph-mon[25908]: mgrmap e16: y(active, since 2m), standbys: x, z 2024-09-17T09:24:30.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:30 smithi003 ceph-mon[25908]: osdmap e21: 3 total, 3 up, 3 in 2024-09-17T09:24:30.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:30 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:30.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:30 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:30.563 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:24:30 smithi183 sudo[40289]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme1n1 2024-09-17T09:24:30.564 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:24:30 smithi183 sudo[40289]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:30.564 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:24:30 smithi183 sudo[40289]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:30.564 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:24:30 smithi183 sudo[40289]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:30.925 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:30 smithi003 sudo[49714]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T09:24:30.925 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:30 smithi003 sudo[49714]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:30.925 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:30 smithi003 sudo[49714]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:30.925 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:30 smithi003 sudo[49714]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:31.218 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:31 smithi003 sudo[49739]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T09:24:31.218 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:31 smithi003 sudo[49739]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:31.218 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:31 smithi003 sudo[49739]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:31.218 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:31 smithi003 sudo[49739]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:31.596 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:24:31.596 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":21,"fsid":"08f126da-74d6-11ef-bceb-c7b262605968","created":"2024-09-17T09:21:22.992100+0000","modified":"2024-09-17T09:24:29.174801+0000","last_up_change":"2024-09-17T09:24:25.823498+0000","last_in_change":"2024-09-17T09:24:07.999623+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T09:24:26.713330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b9e011d8-c4e3-4c7c-ab85-95b39ce8d1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3828533275},{"type":"v1","addr":"172.21.15.3:6803","nonce":3828533275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3828533275},{"type":"v1","addr":"172.21.15.3:6805","nonce":3828533275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3828533275},{"type":"v1","addr":"172.21.15.3:6809","nonce":3828533275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3828533275},{"type":"v1","addr":"172.21.15.3:6807","nonce":3828533275}]},"public_addr":"172.21.15.3:6803/3828533275","cluster_addr":"172.21.15.3:6805/3828533275","heartbeat_back_addr":"172.21.15.3:6809/3828533275","heartbeat_front_addr":"172.21.15.3:6807/3828533275","state":["exists","up"]},{"osd":1,"uuid":"bbfe87f9-1893-411d-a48e-85cfd24590f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":4104131602},{"type":"v1","addr":"172.21.15.77:6801","nonce":4104131602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":4104131602},{"type":"v1","addr":"172.21.15.77:6803","nonce":4104131602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":4104131602},{"type":"v1","addr":"172.21.15.77:6807","nonce":4104131602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":4104131602},{"type":"v1","addr":"172.21.15.77:6805","nonce":4104131602}]},"public_addr":"172.21.15.77:6801/4104131602","cluster_addr":"172.21.15.77:6803/4104131602","heartbeat_back_addr":"172.21.15.77:6807/4104131602","heartbeat_front_addr":"172.21.15.77:6805/4104131602","state":["exists","up"]},{"osd":2,"uuid":"474107a4-c9ef-4cbb-8e7c-3decc0835717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":711224978},{"type":"v1","addr":"172.21.15.183:6801","nonce":711224978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":711224978},{"type":"v1","addr":"172.21.15.183:6803","nonce":711224978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":711224978},{"type":"v1","addr":"172.21.15.183:6807","nonce":711224978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":711224978},{"type":"v1","addr":"172.21.15.183:6805","nonce":711224978}]},"public_addr":"172.21.15.183:6801/711224978","cluster_addr":"172.21.15.183:6803/711224978","heartbeat_back_addr":"172.21.15.183:6807/711224978","heartbeat_front_addr":"172.21.15.183:6805/711224978","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:23:39.186060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:24:01.554476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:24:23.415075+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2998317879":"2024-09-18T09:22:08.212265+0000","172.21.15.3:0/3182416357":"2024-09-18T09:22:08.212265+0000","172.21.15.3:6800/2100925087":"2024-09-18T09:22:08.212265+0000","172.21.15.3:0/2855423234":"2024-09-18T09:22:08.212265+0000","172.21.15.3:6801/1543489825":"2024-09-18T09:21:44.940533+0000","172.21.15.3:6801/2100925087":"2024-09-18T09:22:08.212265+0000","172.21.15.3:0/2104720289":"2024-09-18T09:21:44.940533+0000","172.21.15.3:0/2047567406":"2024-09-18T09:21:44.940533+0000","172.21.15.3:6800/1543489825":"2024-09-18T09:21:44.940533+0000","172.21.15.3:0/262546966":"2024-09-18T09:21:44.940533+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T09:24:31.677 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:31 smithi183 sudo[40292]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T09:24:31.677 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:31 smithi183 sudo[40292]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:31.677 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:31 smithi183 sudo[40292]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:31.677 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:31 smithi183 sudo[40292]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:31.898 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:31 smithi003 ceph-mon[25908]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:31.898 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:31 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:31.898 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:31 smithi003 ceph-mon[25908]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T09:24:31.899 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:31 smithi003 ceph-mon[25908]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T09:24:31.899 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:31 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:24:31.899 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:31 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:24:31.899 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:31 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:24:31.899 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:31 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:31.899 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:31 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:24:31.899 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:31 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:24:31.899 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:31 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:24:31.943 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:31 smithi077 ceph-mon[30659]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:31.943 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:31 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:31.943 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:31 smithi077 ceph-mon[30659]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T09:24:31.943 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:31 smithi077 ceph-mon[30659]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T09:24:31.943 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:31 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:24:31.943 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:31 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:24:31.943 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:31 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:24:31.943 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:31 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:31.943 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:31 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:24:31.943 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:31 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:24:31.944 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:31 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:24:32.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:31 smithi183 sudo[40295]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T09:24:32.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:31 smithi183 sudo[40295]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:32.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:31 smithi183 sudo[40295]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:32.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:31 smithi183 sudo[40295]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:32.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:31 smithi183 ceph-mon[30372]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:32.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:31 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:32.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:31 smithi183 ceph-mon[30372]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T09:24:32.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:31 smithi183 ceph-mon[30372]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T09:24:32.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:31 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:24:32.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:31 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:24:32.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:31 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:24:32.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:31 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:32.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:31 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:24:32.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:31 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:24:32.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:31 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:24:32.195 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:31 smithi077 sudo[39950]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T09:24:32.195 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:31 smithi077 sudo[39950]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:32.195 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:31 smithi077 sudo[39950]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:32.196 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:31 smithi077 sudo[39950]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:32.203 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T09:24:26.713330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 3, 'score_stable': 3, 'optimal_score': 1, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T09:24:32.204 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-17T09:24:32.402 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:24:32.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:32 smithi077 sudo[39953]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T09:24:32.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:32 smithi077 sudo[39953]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:32.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:32 smithi077 sudo[39953]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:32.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:32 smithi077 sudo[39953]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:32.776 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:32 smithi003 ceph-mon[25908]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T09:24:32.776 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:32 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/180755543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T09:24:32.776 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:32 smithi003 ceph-mon[25908]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T09:24:32.776 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:32 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:32.776 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:32 smithi003 ceph-mon[25908]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T09:24:32.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:32 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:24:32.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:32 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:24:32.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:32 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:24:32.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:32 smithi003 ceph-mon[25908]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T09:24:32.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:32 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:32.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:32 smithi003 ceph-mon[25908]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:33.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:32 smithi077 ceph-mon[30659]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T09:24:33.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:32 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/180755543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T09:24:33.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:32 smithi077 ceph-mon[30659]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T09:24:33.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:32 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:33.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:32 smithi077 ceph-mon[30659]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T09:24:33.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:32 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:24:33.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:32 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:24:33.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:32 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:24:33.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:32 smithi077 ceph-mon[30659]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T09:24:33.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:32 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:33.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:32 smithi077 ceph-mon[30659]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:33.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:32 smithi183 ceph-mon[30372]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T09:24:33.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:32 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/180755543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T09:24:33.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:32 smithi183 ceph-mon[30372]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T09:24:33.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:32 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:33.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:32 smithi183 ceph-mon[30372]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T09:24:33.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:32 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:24:33.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:32 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:24:33.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:32 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:24:33.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:32 smithi183 ceph-mon[30372]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T09:24:33.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:32 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:33.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:32 smithi183 ceph-mon[30372]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:34.226 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-17T09:24:34.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:34 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/1130062827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-17T09:24:34.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:34 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/1130062827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-17T09:24:34.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:34 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/1130062827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-17T09:24:34.901 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi003 2024-09-17T09:24:34.901 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi003=foo.a' 2024-09-17T09:24:35.107 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T09:24:35.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:35 smithi183 ceph-mon[30372]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:35.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:35 smithi077 ceph-mon[30659]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:35.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:35 smithi003 ceph-mon[25908]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:36.868 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled rgw.foo.a update... 2024-09-17T09:24:37.464 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:37 smithi183 ceph-mon[30372]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 9.0 KiB/s rd, 479 KiB/s wr, 30 op/s 2024-09-17T09:24:37.464 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:37 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:37.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:37 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:37.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:37 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:37.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:37 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:37.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:37 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:37.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:37 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-17T09:24:37.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:37 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-17T09:24:37.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:37 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:37.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:37 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:37.487 DEBUG:teuthology.orchestra.run.smithi003:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@rgw.foo.a.service 2024-09-17T09:24:37.490 INFO:tasks.cephadm:Adding prometheus.a on smithi077 2024-09-17T09:24:37.490 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph orch apply prometheus '1;smithi077=a' 2024-09-17T09:24:37.575 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:37 smithi003 ceph-mon[25908]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 9.0 KiB/s rd, 479 KiB/s wr, 30 op/s 2024-09-17T09:24:37.576 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:37 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:37.576 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:37 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:37.576 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:37 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:37.576 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:37 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:37.576 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:37 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:37.576 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:37 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-17T09:24:37.576 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:37 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-17T09:24:37.576 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:37 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:37.576 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:37 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:37.682 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T09:24:37.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:37 smithi077 ceph-mon[30659]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 9.0 KiB/s rd, 479 KiB/s wr, 30 op/s 2024-09-17T09:24:37.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:37 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:37.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:37 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:37.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:37 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:37.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:37 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:37.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:37 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:37.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:37 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-17T09:24:37.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:37 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-17T09:24:37.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:37 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:37.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:37 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:38.313 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:38 smithi183 ceph-mon[30372]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi003=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:38.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:38 smithi183 ceph-mon[30372]: Saving service rgw.foo.a spec with placement smithi003=foo.a;count:1 2024-09-17T09:24:38.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:38 smithi183 ceph-mon[30372]: Deploying daemon rgw.foo.a on smithi003 2024-09-17T09:24:38.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:38 smithi183 ceph-mon[30372]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 6.7 KiB/s rd, 359 KiB/s wr, 22 op/s 2024-09-17T09:24:38.559 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:38 smithi003 ceph-mon[25908]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi003=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:38.559 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:38 smithi003 ceph-mon[25908]: Saving service rgw.foo.a spec with placement smithi003=foo.a;count:1 2024-09-17T09:24:38.559 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:38 smithi003 ceph-mon[25908]: Deploying daemon rgw.foo.a on smithi003 2024-09-17T09:24:38.559 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:38 smithi003 ceph-mon[25908]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 6.7 KiB/s rd, 359 KiB/s wr, 22 op/s 2024-09-17T09:24:38.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:38 smithi077 ceph-mon[30659]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi003=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:38.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:38 smithi077 ceph-mon[30659]: Saving service rgw.foo.a spec with placement smithi003=foo.a;count:1 2024-09-17T09:24:38.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:38 smithi077 ceph-mon[30659]: Deploying daemon rgw.foo.a on smithi003 2024-09-17T09:24:38.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:38 smithi077 ceph-mon[30659]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 6.7 KiB/s rd, 359 KiB/s wr, 22 op/s 2024-09-17T09:24:39.258 INFO:journalctl@ceph.rgw.foo.a.smithi003.stdout:Sep 17 09:24:39 smithi003 systemd[1]: Starting Ceph rgw.foo.a for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:24:39.274 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled prometheus update... 2024-09-17T09:24:39.530 INFO:journalctl@ceph.rgw.foo.a.smithi003.stdout:Sep 17 09:24:39 smithi003 podman[50318]: 2024-09-17 09:24:39.167461783 +0000 UTC m=+0.015737465 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T09:24:39.530 INFO:journalctl@ceph.rgw.foo.a.smithi003.stdout:Sep 17 09:24:39 smithi003 podman[50318]: 2024-09-17 09:24:39.268397487 +0000 UTC m=+0.116673155 container create 13fcd40fc00c69c1ccd942472d27916d35ab9b1d51a41cdb0bef2444cc86b5b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-rgw-foo-a, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-17T09:24:39.530 INFO:journalctl@ceph.rgw.foo.a.smithi003.stdout:Sep 17 09:24:39 smithi003 podman[50318]: 2024-09-17 09:24:39.529371487 +0000 UTC m=+0.377647160 container init 13fcd40fc00c69c1ccd942472d27916d35ab9b1d51a41cdb0bef2444cc86b5b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-17T09:24:39.819 DEBUG:teuthology.orchestra.run.smithi077:prometheus.a> sudo journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@prometheus.a.service 2024-09-17T09:24:39.823 INFO:tasks.cephadm:Adding node-exporter.a on smithi003 2024-09-17T09:24:39.823 INFO:tasks.cephadm:Adding node-exporter.b on smithi077 2024-09-17T09:24:39.823 INFO:tasks.cephadm:Adding node-exporter.c on smithi183 2024-09-17T09:24:39.823 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph orch apply node-exporter '3;smithi003=a;smithi077=b;smithi183=c' 2024-09-17T09:24:40.008 INFO:journalctl@ceph.rgw.foo.a.smithi003.stdout:Sep 17 09:24:39 smithi003 podman[50318]: 2024-09-17 09:24:39.534118334 +0000 UTC m=+0.382394003 container start 13fcd40fc00c69c1ccd942472d27916d35ab9b1d51a41cdb0bef2444cc86b5b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T09:24:40.008 INFO:journalctl@ceph.rgw.foo.a.smithi003.stdout:Sep 17 09:24:39 smithi003 bash[50318]: 13fcd40fc00c69c1ccd942472d27916d35ab9b1d51a41cdb0bef2444cc86b5b0 2024-09-17T09:24:40.008 INFO:journalctl@ceph.rgw.foo.a.smithi003.stdout:Sep 17 09:24:39 smithi003 systemd[1]: Started Ceph rgw.foo.a for 08f126da-74d6-11ef-bceb-c7b262605968. 2024-09-17T09:24:40.010 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T09:24:40.444 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:40 smithi183 ceph-mon[30372]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi077=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:40.444 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:40 smithi183 ceph-mon[30372]: Saving service prometheus spec with placement smithi077=a;count:1 2024-09-17T09:24:40.444 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:40 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:40.445 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:40 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:40.445 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:40 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:40.445 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:40 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:40.445 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:40 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:40.445 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:40 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:40.445 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:40 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:40.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:40 smithi077 ceph-mon[30659]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi077=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:40.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:40 smithi077 ceph-mon[30659]: Saving service prometheus spec with placement smithi077=a;count:1 2024-09-17T09:24:40.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:40 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:40.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:40 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:40.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:40 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:40.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:40 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:40.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:40 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:40.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:40 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:40.503 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:40 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:40.732 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:40 smithi003 ceph-mon[25908]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi077=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:40.732 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:40 smithi003 ceph-mon[25908]: Saving service prometheus spec with placement smithi077=a;count:1 2024-09-17T09:24:40.732 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:40 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:40.732 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:40 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:40.733 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:40 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:40.733 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:40 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:40.733 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:40 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:40.733 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:40 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:40.733 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:40 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:41.358 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:41 smithi183 ceph-mon[30372]: Saving service rgw.foo.a spec with placement smithi003=foo.a;count:1 2024-09-17T09:24:41.359 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:41 smithi183 ceph-mon[30372]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 5.4 KiB/s rd, 287 KiB/s wr, 18 op/s 2024-09-17T09:24:41.359 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:41 smithi183 ceph-mon[30372]: osdmap e22: 3 total, 3 up, 3 in 2024-09-17T09:24:41.359 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:41 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/2564625158' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T09:24:41.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:41 smithi077 ceph-mon[30659]: Saving service rgw.foo.a spec with placement smithi003=foo.a;count:1 2024-09-17T09:24:41.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:41 smithi077 ceph-mon[30659]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 5.4 KiB/s rd, 287 KiB/s wr, 18 op/s 2024-09-17T09:24:41.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:41 smithi077 ceph-mon[30659]: osdmap e22: 3 total, 3 up, 3 in 2024-09-17T09:24:41.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:41 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/2564625158' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T09:24:41.756 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled node-exporter update... 2024-09-17T09:24:41.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:41 smithi003 ceph-mon[25908]: Saving service rgw.foo.a spec with placement smithi003=foo.a;count:1 2024-09-17T09:24:41.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:41 smithi003 ceph-mon[25908]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 5.4 KiB/s rd, 287 KiB/s wr, 18 op/s 2024-09-17T09:24:41.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:41 smithi003 ceph-mon[25908]: osdmap e22: 3 total, 3 up, 3 in 2024-09-17T09:24:41.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:41 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/2564625158' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T09:24:42.352 DEBUG:teuthology.orchestra.run.smithi003:node-exporter.a> sudo journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@node-exporter.a.service 2024-09-17T09:24:42.355 DEBUG:teuthology.orchestra.run.smithi077:node-exporter.b> sudo journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@node-exporter.b.service 2024-09-17T09:24:42.358 DEBUG:teuthology.orchestra.run.smithi183:node-exporter.c> sudo journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@node-exporter.c.service 2024-09-17T09:24:42.360 INFO:tasks.cephadm:Adding alertmanager.a on smithi003 2024-09-17T09:24:42.360 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph orch apply alertmanager '1;smithi003=a' 2024-09-17T09:24:42.588 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T09:24:42.707 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:42 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/2564625158' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-17T09:24:42.707 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:42 smithi183 ceph-mon[30372]: osdmap e23: 3 total, 3 up, 3 in 2024-09-17T09:24:42.707 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:42 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:42.707 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:42 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:42.708 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:42 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:42.708 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:42 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:42.708 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:42 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:42.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:42 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/2564625158' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-17T09:24:42.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:42 smithi077 ceph-mon[30659]: osdmap e23: 3 total, 3 up, 3 in 2024-09-17T09:24:42.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:42 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:42.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:42 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:42.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:42 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:42.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:42 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:42.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:42 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:42.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:42 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/2564625158' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-17T09:24:42.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:42 smithi003 ceph-mon[25908]: osdmap e23: 3 total, 3 up, 3 in 2024-09-17T09:24:42.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:42 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:42.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:42 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:42.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:42 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:42.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:42 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:42.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:42 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:43.625 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:43 smithi183 ceph-mon[30372]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi003=a;smithi077=b;smithi183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:43.626 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:43 smithi183 ceph-mon[30372]: Saving service node-exporter spec with placement smithi003=a;smithi077=b;smithi183=c;count:3 2024-09-17T09:24:43.626 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:43 smithi183 ceph-mon[30372]: Checking dashboard <-> RGW credentials 2024-09-17T09:24:43.626 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:43 smithi183 ceph-mon[30372]: pgmap v90: 33 pgs: 10 unknown, 23 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 7.1 KiB/s rd, 359 KiB/s wr, 23 op/s 2024-09-17T09:24:43.626 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:43 smithi183 ceph-mon[30372]: osdmap e24: 3 total, 3 up, 3 in 2024-09-17T09:24:43.626 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:43 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/1770912223' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T09:24:43.626 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:43 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/2564625158' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T09:24:43.626 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:43 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:43.626 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:43 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/1770912223' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T09:24:43.626 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:43 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/2564625158' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T09:24:43.626 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:43 smithi183 ceph-mon[30372]: osdmap e25: 3 total, 3 up, 3 in 2024-09-17T09:24:43.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:43 smithi077 ceph-mon[30659]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi003=a;smithi077=b;smithi183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:43.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:43 smithi077 ceph-mon[30659]: Saving service node-exporter spec with placement smithi003=a;smithi077=b;smithi183=c;count:3 2024-09-17T09:24:43.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:43 smithi077 ceph-mon[30659]: Checking dashboard <-> RGW credentials 2024-09-17T09:24:43.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:43 smithi077 ceph-mon[30659]: pgmap v90: 33 pgs: 10 unknown, 23 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 7.1 KiB/s rd, 359 KiB/s wr, 23 op/s 2024-09-17T09:24:43.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:43 smithi077 ceph-mon[30659]: osdmap e24: 3 total, 3 up, 3 in 2024-09-17T09:24:43.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:43 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/1770912223' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T09:24:43.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:43 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/2564625158' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T09:24:43.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:43 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:43.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:43 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/1770912223' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T09:24:43.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:43 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/2564625158' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T09:24:43.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:43 smithi077 ceph-mon[30659]: osdmap e25: 3 total, 3 up, 3 in 2024-09-17T09:24:43.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:43 smithi003 ceph-mon[25908]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi003=a;smithi077=b;smithi183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:43.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:43 smithi003 ceph-mon[25908]: Saving service node-exporter spec with placement smithi003=a;smithi077=b;smithi183=c;count:3 2024-09-17T09:24:43.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:43 smithi003 ceph-mon[25908]: Checking dashboard <-> RGW credentials 2024-09-17T09:24:43.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:43 smithi003 ceph-mon[25908]: pgmap v90: 33 pgs: 10 unknown, 23 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 7.1 KiB/s rd, 359 KiB/s wr, 23 op/s 2024-09-17T09:24:43.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:43 smithi003 ceph-mon[25908]: osdmap e24: 3 total, 3 up, 3 in 2024-09-17T09:24:43.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:43 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/1770912223' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T09:24:43.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:43 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/2564625158' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T09:24:43.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:43 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:43.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:43 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/1770912223' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T09:24:43.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:43 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/2564625158' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T09:24:43.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:43 smithi003 ceph-mon[25908]: osdmap e25: 3 total, 3 up, 3 in 2024-09-17T09:24:44.254 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled alertmanager update... 2024-09-17T09:24:44.819 DEBUG:teuthology.orchestra.run.smithi003:alertmanager.a> sudo journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@alertmanager.a.service 2024-09-17T09:24:44.821 INFO:tasks.cephadm:Adding grafana.a on smithi077 2024-09-17T09:24:44.821 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph orch apply grafana '1;smithi077=a' 2024-09-17T09:24:45.011 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T09:24:45.381 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:45 smithi183 ceph-mon[30372]: pgmap v93: 65 pgs: 42 unknown, 23 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 767 B/s wr, 1 op/s 2024-09-17T09:24:45.382 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:45 smithi183 ceph-mon[30372]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi003=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:45.382 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:45 smithi183 ceph-mon[30372]: Saving service alertmanager spec with placement smithi003=a;count:1 2024-09-17T09:24:45.382 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:45 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:45.382 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:45 smithi183 ceph-mon[30372]: osdmap e26: 3 total, 3 up, 3 in 2024-09-17T09:24:45.382 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:45 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/2564625158' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T09:24:45.382 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:45 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/1770912223' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T09:24:45.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:45 smithi003 ceph-mon[25908]: pgmap v93: 65 pgs: 42 unknown, 23 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 767 B/s wr, 1 op/s 2024-09-17T09:24:45.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:45 smithi003 ceph-mon[25908]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi003=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:45.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:45 smithi003 ceph-mon[25908]: Saving service alertmanager spec with placement smithi003=a;count:1 2024-09-17T09:24:45.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:45 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:45.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:45 smithi003 ceph-mon[25908]: osdmap e26: 3 total, 3 up, 3 in 2024-09-17T09:24:45.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:45 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/2564625158' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T09:24:45.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:45 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/1770912223' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T09:24:45.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:45 smithi077 ceph-mon[30659]: pgmap v93: 65 pgs: 42 unknown, 23 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 767 B/s wr, 1 op/s 2024-09-17T09:24:45.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:45 smithi077 ceph-mon[30659]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi003=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:45.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:45 smithi077 ceph-mon[30659]: Saving service alertmanager spec with placement smithi003=a;count:1 2024-09-17T09:24:45.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:45 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:45.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:45 smithi077 ceph-mon[30659]: osdmap e26: 3 total, 3 up, 3 in 2024-09-17T09:24:45.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:45 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/2564625158' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T09:24:45.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:45 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/1770912223' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T09:24:46.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:46 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/2564625158' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T09:24:46.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:46 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/1770912223' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T09:24:46.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:46 smithi183 ceph-mon[30372]: osdmap e27: 3 total, 3 up, 3 in 2024-09-17T09:24:46.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:46 smithi183 ceph-mon[30372]: pgmap v96: 97 pgs: 9 creating+peering, 44 unknown, 44 active+clean; 578 KiB data, 85 MiB used, 268 GiB / 268 GiB avail; 511 B/s rd, 255 B/s wr, 0 op/s 2024-09-17T09:24:46.648 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled grafana update... 2024-09-17T09:24:46.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:46 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/2564625158' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T09:24:46.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:46 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/1770912223' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T09:24:46.654 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:46 smithi003 ceph-mon[25908]: osdmap e27: 3 total, 3 up, 3 in 2024-09-17T09:24:46.654 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:46 smithi003 ceph-mon[25908]: pgmap v96: 97 pgs: 9 creating+peering, 44 unknown, 44 active+clean; 578 KiB data, 85 MiB used, 268 GiB / 268 GiB avail; 511 B/s rd, 255 B/s wr, 0 op/s 2024-09-17T09:24:46.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:46 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/2564625158' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T09:24:46.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:46 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/1770912223' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T09:24:46.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:46 smithi077 ceph-mon[30659]: osdmap e27: 3 total, 3 up, 3 in 2024-09-17T09:24:46.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:46 smithi077 ceph-mon[30659]: pgmap v96: 97 pgs: 9 creating+peering, 44 unknown, 44 active+clean; 578 KiB data, 85 MiB used, 268 GiB / 268 GiB avail; 511 B/s rd, 255 B/s wr, 0 op/s 2024-09-17T09:24:47.316 DEBUG:teuthology.orchestra.run.smithi077:grafana.a> sudo journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@grafana.a.service 2024-09-17T09:24:47.319 INFO:tasks.cephadm:Setting up client nodes... 2024-09-17T09:24:47.319 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-17T09:24:47.528 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:24:47.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:47 smithi183 ceph-mon[30372]: osdmap e28: 3 total, 3 up, 3 in 2024-09-17T09:24:47.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:47 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/2880035586' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T09:24:47.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:47 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/2170538082' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T09:24:47.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:47 smithi183 ceph-mon[30372]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T09:24:47.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:47 smithi183 ceph-mon[30372]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi077=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:47.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:47 smithi183 ceph-mon[30372]: Saving service grafana spec with placement smithi077=a;count:1 2024-09-17T09:24:47.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:47 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:47.643 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:47 smithi003 ceph-mon[25908]: osdmap e28: 3 total, 3 up, 3 in 2024-09-17T09:24:47.643 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:47 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/2880035586' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T09:24:47.643 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:47 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/2170538082' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T09:24:47.643 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:47 smithi003 ceph-mon[25908]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T09:24:47.643 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:47 smithi003 ceph-mon[25908]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi077=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:47.643 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:47 smithi003 ceph-mon[25908]: Saving service grafana spec with placement smithi077=a;count:1 2024-09-17T09:24:47.643 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:47 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:47.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:47 smithi077 ceph-mon[30659]: osdmap e28: 3 total, 3 up, 3 in 2024-09-17T09:24:47.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:47 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/2880035586' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T09:24:47.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:47 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/2170538082' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T09:24:47.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:47 smithi077 ceph-mon[30659]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T09:24:47.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:47 smithi077 ceph-mon[30659]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi077=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:47.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:47 smithi077 ceph-mon[30659]: Saving service grafana spec with placement smithi077=a;count:1 2024-09-17T09:24:47.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:47 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:48.715 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:48 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/2170538082' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T09:24:48.715 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:48 smithi003 ceph-mon[25908]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T09:24:48.715 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:48 smithi003 ceph-mon[25908]: osdmap e29: 3 total, 3 up, 3 in 2024-09-17T09:24:48.715 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:48 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/2880035586' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T09:24:48.715 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:48 smithi003 ceph-mon[25908]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T09:24:48.715 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:48 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/2170538082' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T09:24:48.716 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:48 smithi003 ceph-mon[25908]: pgmap v99: 129 pgs: 28 creating+peering, 13 unknown, 88 active+clean; 578 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-17T09:24:48.716 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:48 smithi003 ceph-mon[25908]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T09:24:48.716 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:48 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/2170538082' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T09:24:48.716 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:48 smithi003 ceph-mon[25908]: osdmap e30: 3 total, 3 up, 3 in 2024-09-17T09:24:48.720 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:48 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/2170538082' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T09:24:48.720 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:48 smithi183 ceph-mon[30372]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T09:24:48.721 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:48 smithi183 ceph-mon[30372]: osdmap e29: 3 total, 3 up, 3 in 2024-09-17T09:24:48.721 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:48 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/2880035586' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T09:24:48.721 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:48 smithi183 ceph-mon[30372]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T09:24:48.721 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:48 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/2170538082' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T09:24:48.721 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:48 smithi183 ceph-mon[30372]: pgmap v99: 129 pgs: 28 creating+peering, 13 unknown, 88 active+clean; 578 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-17T09:24:48.721 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:48 smithi183 ceph-mon[30372]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T09:24:48.721 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:48 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/2170538082' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T09:24:48.721 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:48 smithi183 ceph-mon[30372]: osdmap e30: 3 total, 3 up, 3 in 2024-09-17T09:24:48.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:48 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/2170538082' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T09:24:48.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:48 smithi077 ceph-mon[30659]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T09:24:48.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:48 smithi077 ceph-mon[30659]: osdmap e29: 3 total, 3 up, 3 in 2024-09-17T09:24:48.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:48 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/2880035586' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T09:24:48.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:48 smithi077 ceph-mon[30659]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T09:24:48.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:48 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/2170538082' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T09:24:48.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:48 smithi077 ceph-mon[30659]: pgmap v99: 129 pgs: 28 creating+peering, 13 unknown, 88 active+clean; 578 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-17T09:24:48.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:48 smithi077 ceph-mon[30659]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T09:24:48.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:48 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/2170538082' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T09:24:48.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:48 smithi077 ceph-mon[30659]: osdmap e30: 3 total, 3 up, 3 in 2024-09-17T09:24:49.480 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-09-17T09:24:49.480 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQDhSulm085gHBAARNbd5LV5xR0RCSvRELsiew== 2024-09-17T09:24:49.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:49 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:49.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:49 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:49.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:49 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:49.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:49 smithi003 ceph-mon[25908]: Deploying daemon prometheus.a on smithi077 2024-09-17T09:24:49.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:49 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/441556967' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:24:49.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:49 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/441556967' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T09:24:49.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:49 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:49.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:49 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:49.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:49 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:49.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:49 smithi183 ceph-mon[30372]: Deploying daemon prometheus.a on smithi077 2024-09-17T09:24:49.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:49 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/441556967' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:24:49.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:49 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/441556967' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T09:24:50.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:49 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:50.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:49 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:50.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:49 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:24:50.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:49 smithi077 ceph-mon[30659]: Deploying daemon prometheus.a on smithi077 2024-09-17T09:24:50.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:49 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/441556967' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:24:50.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:49 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/441556967' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T09:24:50.077 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-17T09:24:50.077 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-17T09:24:50.078 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T09:24:50.119 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-17T09:24:50.388 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:24:50.768 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:50 smithi077 ceph-mon[30659]: pgmap v101: 129 pgs: 19 creating+peering, 110 active+clean; 579 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 35 KiB/s rd, 2.7 KiB/s wr, 84 op/s 2024-09-17T09:24:50.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:50 smithi183 ceph-mon[30372]: pgmap v101: 129 pgs: 19 creating+peering, 110 active+clean; 579 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 35 KiB/s rd, 2.7 KiB/s wr, 84 op/s 2024-09-17T09:24:51.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:50 smithi003 ceph-mon[25908]: pgmap v101: 129 pgs: 19 creating+peering, 110 active+clean; 579 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 35 KiB/s rd, 2.7 KiB/s wr, 84 op/s 2024-09-17T09:24:52.593 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:52 smithi077 ceph-mon[30659]: pgmap v102: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 77 KiB/s rd, 6.0 KiB/s wr, 187 op/s 2024-09-17T09:24:52.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:52 smithi003 ceph-mon[25908]: pgmap v102: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 77 KiB/s rd, 6.0 KiB/s wr, 187 op/s 2024-09-17T09:24:52.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:52 smithi183 ceph-mon[30372]: pgmap v102: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 77 KiB/s rd, 6.0 KiB/s wr, 187 op/s 2024-09-17T09:24:52.924 INFO:teuthology.orchestra.run.smithi077.stdout:[client.1] 2024-09-17T09:24:52.924 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQDkSulmOozONhAARdwSPmPn0ST+0zI8dYC7IQ== 2024-09-17T09:24:53.709 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:53 smithi077 ceph-mon[30659]: from='client.? 172.21.15.77:0/2330786775' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:24:53.709 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:53 smithi077 ceph-mon[30659]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:24:53.709 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:53 smithi077 ceph-mon[30659]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T09:24:53.733 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-17T09:24:53.733 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-17T09:24:53.733 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-17T09:24:53.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:53 smithi003 ceph-mon[25908]: from='client.? 172.21.15.77:0/2330786775' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:24:53.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:53 smithi003 ceph-mon[25908]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:24:53.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:53 smithi003 ceph-mon[25908]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T09:24:53.774 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.2 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-17T09:24:53.803 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:53 smithi183 ceph-mon[30372]: from='client.? 172.21.15.77:0/2330786775' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:24:53.803 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:53 smithi183 ceph-mon[30372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:24:53.803 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:53 smithi183 ceph-mon[30372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T09:24:53.980 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.c/config 2024-09-17T09:24:54.345 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:54 smithi183 ceph-mon[30372]: pgmap v103: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 59 KiB/s rd, 4.6 KiB/s wr, 142 op/s 2024-09-17T09:24:54.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:54 smithi077 ceph-mon[30659]: pgmap v103: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 59 KiB/s rd, 4.6 KiB/s wr, 142 op/s 2024-09-17T09:24:54.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:54 smithi003 ceph-mon[25908]: pgmap v103: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 59 KiB/s rd, 4.6 KiB/s wr, 142 op/s 2024-09-17T09:24:55.794 INFO:teuthology.orchestra.run.smithi183.stdout:[client.2] 2024-09-17T09:24:55.794 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDnSulmJ2AILxAA7k6/xfLlz6tp/pXiETjviA== 2024-09-17T09:24:56.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:55 smithi183 ceph-mon[30372]: from='client.? 172.21.15.183:0/3879392367' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:24:56.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:55 smithi183 ceph-mon[30372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:24:56.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:55 smithi183 ceph-mon[30372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T09:24:56.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:55 smithi077 ceph-mon[30659]: from='client.? 172.21.15.183:0/3879392367' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:24:56.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:55 smithi077 ceph-mon[30659]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:24:56.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:55 smithi077 ceph-mon[30659]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T09:24:56.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:55 smithi003 ceph-mon[25908]: from='client.? 172.21.15.183:0/3879392367' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:24:56.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:55 smithi003 ceph-mon[25908]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:24:56.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:55 smithi003 ceph-mon[25908]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T09:24:56.341 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-17T09:24:56.341 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.client.2.keyring 2024-09-17T09:24:56.341 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-17T09:24:56.418 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T09:24:56.418 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-17T09:24:56.418 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-17T09:24:56.630 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:24:57.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:56 smithi077 ceph-mon[30659]: pgmap v104: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 51 KiB/s rd, 3.9 KiB/s wr, 125 op/s 2024-09-17T09:24:57.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:56 smithi003 ceph-mon[25908]: pgmap v104: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 51 KiB/s rd, 3.9 KiB/s wr, 125 op/s 2024-09-17T09:24:57.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:56 smithi183 ceph-mon[30372]: pgmap v104: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 51 KiB/s rd, 3.9 KiB/s wr, 125 op/s 2024-09-17T09:24:58.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:58 smithi077 ceph-mon[30659]: pgmap v105: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 111 op/s 2024-09-17T09:24:58.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:58 smithi003 ceph-mon[25908]: pgmap v105: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 111 op/s 2024-09-17T09:24:58.549 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:24:58.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:58 smithi183 ceph-mon[30372]: pgmap v105: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 111 op/s 2024-09-17T09:24:59.189 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":16,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":1869758615},{"type":"v1","addr":"172.21.15.3:6801","nonce":1869758615}]},"active_addr":"172.21.15.3:6801/1869758615","active_change":"2024-09-17T09:22:08.212375+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24104,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24106,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.3:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":2019317465}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":1409999168}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":707821748}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":4170366017}]}]} 2024-09-17T09:24:59.196 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-17T09:24:59.197 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-17T09:24:59.197 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-17T09:24:59.398 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:24:59.504 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:24:59 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/3463715599' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-17T09:24:59.504 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:24:59 smithi077 systemd[1]: Starting Ceph prometheus.a for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:24:59.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:24:59 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/3463715599' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-17T09:24:59.563 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:24:59 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/3463715599' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-17T09:24:59.891 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:24:59 smithi077 podman[40933]: 2024-09-17 09:24:59.498937859 +0000 UTC m=+0.016269596 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-17T09:24:59.891 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:24:59 smithi077 podman[40933]: 2024-09-17 09:24:59.699093558 +0000 UTC m=+0.216425289 container create baa612d113edfff83e522ed0f700c7cbbb4f1445f05656a1f1ff88155fdf8c13 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T09:25:00.252 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:24:59 smithi077 podman[40933]: 2024-09-17 09:24:59.994660924 +0000 UTC m=+0.511992659 container init baa612d113edfff83e522ed0f700c7cbbb4f1445f05656a1f1ff88155fdf8c13 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T09:25:00.253 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:24:59 smithi077 podman[40933]: 2024-09-17 09:24:59.999318308 +0000 UTC m=+0.516650036 container start baa612d113edfff83e522ed0f700c7cbbb4f1445f05656a1f1ff88155fdf8c13 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T09:25:00.253 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:25:00.024Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-17T09:25:00.253 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:25:00.024Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-17T09:25:00.253 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:25:00.024Z caller=main.go:570 level=info host_details="(Linux 5.14.0-508.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024 x86_64 smithi077 (none))" 2024-09-17T09:25:00.253 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:25:00.024Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-17T09:25:00.253 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:25:00.024Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-17T09:25:00.253 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:25:00.025Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-17T09:25:00.253 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:25:00.025Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-17T09:25:00.253 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:25:00.026Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-17T09:25:00.253 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:25:00.026Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-17T09:25:00.253 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:25:00.028Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-17T09:25:00.254 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:25:00.028Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.506µs 2024-09-17T09:25:00.254 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:25:00.028Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-17T09:25:00.254 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:25:00.029Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-17T09:25:00.254 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:25:00.029Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=15.516µs wal_replay_duration=1.492475ms wbl_replay_duration=138ns total_replay_duration=1.525889ms 2024-09-17T09:25:00.254 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:25:00.030Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-17T09:25:00.254 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:25:00.030Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-17T09:25:00.254 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:25:00.030Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-17T09:25:00.254 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:25:00.051Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=20.266668ms db_storage=768ns remote_storage=1.78µs web_handler=356ns query_engine=561ns scrape=467.379µs scrape_sd=87.88µs notify=974ns notify_sd=2.431µs rules=19.446202ms tracing=6.334µs 2024-09-17T09:25:00.254 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:25:00.051Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-17T09:25:00.254 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:25:00.051Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-17T09:25:00.254 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 bash[40933]: baa612d113edfff83e522ed0f700c7cbbb4f1445f05656a1f1ff88155fdf8c13 2024-09-17T09:25:00.255 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:25:00 smithi077 systemd[1]: Started Ceph prometheus.a for 08f126da-74d6-11ef-bceb-c7b262605968. 2024-09-17T09:25:01.284 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:25:01.284 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":30,"fsid":"08f126da-74d6-11ef-bceb-c7b262605968","created":"2024-09-17T09:21:22.992100+0000","modified":"2024-09-17T09:24:48.351899+0000","last_up_change":"2024-09-17T09:24:25.823498+0000","last_in_change":"2024-09-17T09:24:07.999623+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T09:24:26.713330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-17T09:24:40.001743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-09-17T09:24:41.742918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-09-17T09:24:43.778635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.meta","create_time":"2024-09-17T09:24:45.856573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b9e011d8-c4e3-4c7c-ab85-95b39ce8d1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3828533275},{"type":"v1","addr":"172.21.15.3:6803","nonce":3828533275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3828533275},{"type":"v1","addr":"172.21.15.3:6805","nonce":3828533275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3828533275},{"type":"v1","addr":"172.21.15.3:6809","nonce":3828533275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3828533275},{"type":"v1","addr":"172.21.15.3:6807","nonce":3828533275}]},"public_addr":"172.21.15.3:6803/3828533275","cluster_addr":"172.21.15.3:6805/3828533275","heartbeat_back_addr":"172.21.15.3:6809/3828533275","heartbeat_front_addr":"172.21.15.3:6807/3828533275","state":["exists","up"]},{"osd":1,"uuid":"bbfe87f9-1893-411d-a48e-85cfd24590f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":4104131602},{"type":"v1","addr":"172.21.15.77:6801","nonce":4104131602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":4104131602},{"type":"v1","addr":"172.21.15.77:6803","nonce":4104131602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":4104131602},{"type":"v1","addr":"172.21.15.77:6807","nonce":4104131602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":4104131602},{"type":"v1","addr":"172.21.15.77:6805","nonce":4104131602}]},"public_addr":"172.21.15.77:6801/4104131602","cluster_addr":"172.21.15.77:6803/4104131602","heartbeat_back_addr":"172.21.15.77:6807/4104131602","heartbeat_front_addr":"172.21.15.77:6805/4104131602","state":["exists","up"]},{"osd":2,"uuid":"474107a4-c9ef-4cbb-8e7c-3decc0835717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":711224978},{"type":"v1","addr":"172.21.15.183:6801","nonce":711224978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":711224978},{"type":"v1","addr":"172.21.15.183:6803","nonce":711224978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":711224978},{"type":"v1","addr":"172.21.15.183:6807","nonce":711224978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":711224978},{"type":"v1","addr":"172.21.15.183:6805","nonce":711224978}]},"public_addr":"172.21.15.183:6801/711224978","cluster_addr":"172.21.15.183:6803/711224978","heartbeat_back_addr":"172.21.15.183:6807/711224978","heartbeat_front_addr":"172.21.15.183:6805/711224978","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:23:39.186060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:24:01.554476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:24:23.415075+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2998317879":"2024-09-18T09:22:08.212265+0000","172.21.15.3:0/3182416357":"2024-09-18T09:22:08.212265+0000","172.21.15.3:6800/2100925087":"2024-09-18T09:22:08.212265+0000","172.21.15.3:0/2855423234":"2024-09-18T09:22:08.212265+0000","172.21.15.3:6801/1543489825":"2024-09-18T09:21:44.940533+0000","172.21.15.3:6801/2100925087":"2024-09-18T09:22:08.212265+0000","172.21.15.3:0/2104720289":"2024-09-18T09:21:44.940533+0000","172.21.15.3:0/2047567406":"2024-09-18T09:21:44.940533+0000","172.21.15.3:6800/1543489825":"2024-09-18T09:21:44.940533+0000","172.21.15.3:0/262546966":"2024-09-18T09:21:44.940533+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T09:25:01.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:25:01.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:25:01.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:25:01.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:01 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-17T09:25:01.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:01 smithi077 ceph-mon[30659]: pgmap v106: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 38 KiB/s rd, 2.9 KiB/s wr, 93 op/s 2024-09-17T09:25:01.503 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:01 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: ignoring --setuser ceph since I am not root 2024-09-17T09:25:01.503 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:01 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: ignoring --setgroup ceph since I am not root 2024-09-17T09:25:01.503 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:01 smithi077 ceph-mgr[32111]: -- 172.21.15.77:0/3004715289 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560d43504d00 con 0x560d434d9000 2024-09-17T09:25:01.503 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:01 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:01.328+0000 7f50f9baf040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T09:25:01.503 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:01 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:01.386+0000 7f50f9baf040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T09:25:01.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:01 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:25:01.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:01 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:25:01.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:01 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:25:01.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:01 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-17T09:25:01.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:01 smithi003 ceph-mon[25908]: pgmap v106: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 38 KiB/s rd, 2.9 KiB/s wr, 93 op/s 2024-09-17T09:25:01.508 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:01 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: ignoring --setuser ceph since I am not root 2024-09-17T09:25:01.509 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:01 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: ignoring --setgroup ceph since I am not root 2024-09-17T09:25:01.509 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:01 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:01.326+0000 7f9d82fb2040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T09:25:01.509 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:01 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:01.385+0000 7f9d82fb2040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T09:25:01.525 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:01 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:25:01.525 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:01 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:25:01.525 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:01 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' 2024-09-17T09:25:01.525 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:01 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-17T09:25:01.525 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:01 smithi183 ceph-mon[30372]: pgmap v106: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 38 KiB/s rd, 2.9 KiB/s wr, 93 op/s 2024-09-17T09:25:01.525 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:01 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: ignoring --setuser ceph since I am not root 2024-09-17T09:25:01.525 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:01 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: ignoring --setgroup ceph since I am not root 2024-09-17T09:25:01.526 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:01 smithi183 ceph-mgr[32447]: -- 172.21.15.183:0/2345365497 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559869c4ed00 con 0x559869c23400 2024-09-17T09:25:01.526 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:01 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:01.321+0000 7f0d722f6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T09:25:01.526 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:01 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:01.378+0000 7f0d722f6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T09:25:01.526 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:01 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:01.523+0000 7f0d722f6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T09:25:01.762 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:01 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:01.539+0000 7f9d82fb2040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T09:25:01.763 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:01 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:01.609+0000 7f9d82fb2040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T09:25:01.763 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:01 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:01.760+0000 7f9d82fb2040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:25:01.814 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:01 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:01.589+0000 7f0d722f6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T09:25:01.814 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:01 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:01.730+0000 7f0d722f6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:25:01.883 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:01 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:01.535+0000 7f50f9baf040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T09:25:01.883 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:01 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:01.600+0000 7f50f9baf040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T09:25:01.883 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:01 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:01.743+0000 7f50f9baf040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:25:01.985 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-17T09:25:01.986 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-17T09:25:02.169 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:01 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:01.868+0000 7f0d722f6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T09:25:02.172 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:02 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:02.167+0000 7f0d722f6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T09:25:02.175 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:01 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:01.882+0000 7f50f9baf040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T09:25:02.175 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:01 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:01.906+0000 7f9d82fb2040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T09:25:02.191 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:25:02.428 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:02 smithi077 ceph-mon[30659]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-17T09:25:02.428 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:02 smithi077 ceph-mon[30659]: mgrmap e17: y(active, since 2m), standbys: x, z 2024-09-17T09:25:02.428 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:02 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/2002899481' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T09:25:02.428 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:02 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:02.185+0000 7f50f9baf040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T09:25:02.428 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:02 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:02.245+0000 7f50f9baf040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T09:25:02.428 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:02 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:02.426+0000 7f50f9baf040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T09:25:02.450 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:02 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:02.241+0000 7f9d82fb2040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T09:25:02.450 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:02 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:02.304+0000 7f9d82fb2040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T09:25:02.450 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:02 smithi003 ceph-mon[25908]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-17T09:25:02.450 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:02 smithi003 ceph-mon[25908]: mgrmap e17: y(active, since 2m), standbys: x, z 2024-09-17T09:25:02.450 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:02 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/2002899481' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T09:25:02.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:02 smithi183 ceph-mon[30372]: from='mgr.14150 172.21.15.3:0/2810080520' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-17T09:25:02.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:02 smithi183 ceph-mon[30372]: mgrmap e17: y(active, since 2m), standbys: x, z 2024-09-17T09:25:02.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:02 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/2002899481' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T09:25:02.564 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:02 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:02.228+0000 7f0d722f6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T09:25:02.564 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:02 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:02.408+0000 7f0d722f6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T09:25:02.707 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:02 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:02.502+0000 7f9d82fb2040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T09:25:03.564 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:03 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:03.263+0000 7f0d722f6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T09:25:03.564 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:03 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:03.329+0000 7f0d722f6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T09:25:03.752 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:03 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:03.278+0000 7f50f9baf040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T09:25:03.752 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:03 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:03.342+0000 7f50f9baf040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T09:25:03.755 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:03 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:03.431+0000 7f9d82fb2040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T09:25:03.755 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:03 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:03.501+0000 7f9d82fb2040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T09:25:04.021 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:25:04.021 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":30,"fsid":"08f126da-74d6-11ef-bceb-c7b262605968","created":"2024-09-17T09:21:22.992100+0000","modified":"2024-09-17T09:24:48.351899+0000","last_up_change":"2024-09-17T09:24:25.823498+0000","last_in_change":"2024-09-17T09:24:07.999623+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T09:24:26.713330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-17T09:24:40.001743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-09-17T09:24:41.742918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-09-17T09:24:43.778635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.meta","create_time":"2024-09-17T09:24:45.856573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b9e011d8-c4e3-4c7c-ab85-95b39ce8d1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3828533275},{"type":"v1","addr":"172.21.15.3:6803","nonce":3828533275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3828533275},{"type":"v1","addr":"172.21.15.3:6805","nonce":3828533275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3828533275},{"type":"v1","addr":"172.21.15.3:6809","nonce":3828533275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3828533275},{"type":"v1","addr":"172.21.15.3:6807","nonce":3828533275}]},"public_addr":"172.21.15.3:6803/3828533275","cluster_addr":"172.21.15.3:6805/3828533275","heartbeat_back_addr":"172.21.15.3:6809/3828533275","heartbeat_front_addr":"172.21.15.3:6807/3828533275","state":["exists","up"]},{"osd":1,"uuid":"bbfe87f9-1893-411d-a48e-85cfd24590f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":4104131602},{"type":"v1","addr":"172.21.15.77:6801","nonce":4104131602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":4104131602},{"type":"v1","addr":"172.21.15.77:6803","nonce":4104131602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":4104131602},{"type":"v1","addr":"172.21.15.77:6807","nonce":4104131602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":4104131602},{"type":"v1","addr":"172.21.15.77:6805","nonce":4104131602}]},"public_addr":"172.21.15.77:6801/4104131602","cluster_addr":"172.21.15.77:6803/4104131602","heartbeat_back_addr":"172.21.15.77:6807/4104131602","heartbeat_front_addr":"172.21.15.77:6805/4104131602","state":["exists","up"]},{"osd":2,"uuid":"474107a4-c9ef-4cbb-8e7c-3decc0835717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":711224978},{"type":"v1","addr":"172.21.15.183:6801","nonce":711224978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":711224978},{"type":"v1","addr":"172.21.15.183:6803","nonce":711224978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":711224978},{"type":"v1","addr":"172.21.15.183:6807","nonce":711224978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":711224978},{"type":"v1","addr":"172.21.15.183:6805","nonce":711224978}]},"public_addr":"172.21.15.183:6801/711224978","cluster_addr":"172.21.15.183:6803/711224978","heartbeat_back_addr":"172.21.15.183:6807/711224978","heartbeat_front_addr":"172.21.15.183:6805/711224978","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:23:39.186060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:24:01.554476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:24:23.415075+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2998317879":"2024-09-18T09:22:08.212265+0000","172.21.15.3:0/3182416357":"2024-09-18T09:22:08.212265+0000","172.21.15.3:6800/2100925087":"2024-09-18T09:22:08.212265+0000","172.21.15.3:0/2855423234":"2024-09-18T09:22:08.212265+0000","172.21.15.3:6801/1543489825":"2024-09-18T09:21:44.940533+0000","172.21.15.3:6801/2100925087":"2024-09-18T09:22:08.212265+0000","172.21.15.3:0/2104720289":"2024-09-18T09:21:44.940533+0000","172.21.15.3:0/2047567406":"2024-09-18T09:21:44.940533+0000","172.21.15.3:6800/1543489825":"2024-09-18T09:21:44.940533+0000","172.21.15.3:0/262546966":"2024-09-18T09:21:44.940533+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T09:25:04.284 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:04 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/194118569' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T09:25:04.285 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:04 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:04.029+0000 7f0d722f6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T09:25:04.285 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:04 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:04.144+0000 7f0d722f6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T09:25:04.300 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:04 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/194118569' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T09:25:04.300 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:04 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:04.045+0000 7f50f9baf040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T09:25:04.300 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:04 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:04.159+0000 7f50f9baf040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T09:25:04.325 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:04 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/194118569' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T09:25:04.325 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:04 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:04.266+0000 7f9d82fb2040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T09:25:04.558 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:04 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:04.282+0000 7f0d722f6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:25:04.558 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:04 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:04.344+0000 7f0d722f6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T09:25:04.558 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:04 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:04.490+0000 7f0d722f6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T09:25:04.558 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:04 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:04.556+0000 7f0d722f6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T09:25:04.573 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:04 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:04.298+0000 7f50f9baf040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:25:04.573 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:04 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:04.360+0000 7f50f9baf040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T09:25:04.573 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:04 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:04.506+0000 7f50f9baf040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T09:25:04.573 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:04 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:04.571+0000 7f50f9baf040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T09:25:04.592 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:04 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:04.386+0000 7f9d82fb2040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T09:25:04.592 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:04 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:04.534+0000 7f9d82fb2040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:25:04.615 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-17T09:25:04.615 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-17T09:25:04.616 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-17T09:25:04.814 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:04 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:04.616+0000 7f0d722f6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T09:25:04.814 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:04 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:04.770+0000 7f0d722f6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T09:25:04.849 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:04 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:04.631+0000 7f50f9baf040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T09:25:04.849 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:04 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:04.783+0000 7f50f9baf040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T09:25:04.849 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:04 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:04.847+0000 7f50f9baf040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T09:25:04.857 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:25:04.859 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:25:04.877 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:25:04.913 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:04 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:04.598+0000 7f9d82fb2040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T09:25:04.913 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:04 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:04.819+0000 7f9d82fb2040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T09:25:04.913 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:04 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:04.911+0000 7f9d82fb2040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T09:25:05.097 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:04 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:04.835+0000 7f0d722f6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T09:25:05.097 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:04 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:04.899+0000 7f0d722f6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T09:25:05.097 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:04 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:04.979+0000 7f0d722f6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T09:25:05.097 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:05 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:05.037+0000 7f0d722f6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T09:25:05.097 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:05 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:05.095+0000 7f0d722f6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T09:25:05.111 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:04 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:04.912+0000 7f50f9baf040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T09:25:05.111 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:04 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:04.994+0000 7f50f9baf040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T09:25:05.112 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:05 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:05.052+0000 7f50f9baf040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T09:25:05.112 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:05 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:05.110+0000 7f50f9baf040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T09:25:05.214 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:04 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:04.973+0000 7f9d82fb2040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T09:25:05.214 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:05 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:05.144+0000 7f9d82fb2040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T09:25:05.214 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:05 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:05.212+0000 7f9d82fb2040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T09:25:05.500 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:05 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:05.281+0000 7f9d82fb2040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T09:25:05.501 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:05 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:05.368+0000 7f9d82fb2040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T09:25:05.501 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:05 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:05.436+0000 7f9d82fb2040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T09:25:05.501 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:05 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:05.497+0000 7f9d82fb2040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T09:25:06.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:05 smithi077 ceph-mon[30659]: Standby manager daemon z restarted 2024-09-17T09:25:06.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:05 smithi077 ceph-mon[30659]: Standby manager daemon z started 2024-09-17T09:25:06.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:05 smithi077 ceph-mon[30659]: from='mgr.? 172.21.15.183:0/1581754822' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-17T09:25:06.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:05 smithi077 ceph-mon[30659]: from='mgr.? 172.21.15.183:0/1581754822' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T09:25:06.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:05 smithi077 ceph-mon[30659]: from='mgr.? 172.21.15.183:0/1581754822' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-17T09:25:06.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:05 smithi077 ceph-mon[30659]: from='mgr.? 172.21.15.183:0/1581754822' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T09:25:06.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:05 smithi077 ceph-mon[30659]: Standby manager daemon x restarted 2024-09-17T09:25:06.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:05 smithi077 ceph-mon[30659]: Standby manager daemon x started 2024-09-17T09:25:06.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:05 smithi077 ceph-mon[30659]: from='mgr.? 172.21.15.77:0/1058997546' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T09:25:06.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:05 smithi077 ceph-mon[30659]: from='mgr.? 172.21.15.77:0/1058997546' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T09:25:06.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:05 smithi077 ceph-mon[30659]: from='mgr.? 172.21.15.77:0/1058997546' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T09:25:06.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:05 smithi077 ceph-mon[30659]: from='mgr.? 172.21.15.77:0/1058997546' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T09:25:06.253 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:05 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:05.881+0000 7f50f9baf040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T09:25:06.253 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:05 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: [17/Sep/2024:09:25:05] ENGINE Bus STARTING 2024-09-17T09:25:06.254 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:05 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: CherryPy Checker: 2024-09-17T09:25:06.254 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:05 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: The Application mounted at '' has an empty config. 2024-09-17T09:25:06.254 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:05 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: 2024-09-17T09:25:06.254 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:05 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: [17/Sep/2024:09:25:05] ENGINE Serving on http://:::9283 2024-09-17T09:25:06.254 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:25:05 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x[32088]: [17/Sep/2024:09:25:05] ENGINE Bus STARTED 2024-09-17T09:25:06.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:05 smithi003 ceph-mon[25908]: Standby manager daemon z restarted 2024-09-17T09:25:06.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:05 smithi003 ceph-mon[25908]: Standby manager daemon z started 2024-09-17T09:25:06.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:05 smithi003 ceph-mon[25908]: from='mgr.? 172.21.15.183:0/1581754822' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-17T09:25:06.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:05 smithi003 ceph-mon[25908]: from='mgr.? 172.21.15.183:0/1581754822' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T09:25:06.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:05 smithi003 ceph-mon[25908]: from='mgr.? 172.21.15.183:0/1581754822' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-17T09:25:06.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:05 smithi003 ceph-mon[25908]: from='mgr.? 172.21.15.183:0/1581754822' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T09:25:06.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:05 smithi003 ceph-mon[25908]: Standby manager daemon x restarted 2024-09-17T09:25:06.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:05 smithi003 ceph-mon[25908]: Standby manager daemon x started 2024-09-17T09:25:06.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:05 smithi003 ceph-mon[25908]: from='mgr.? 172.21.15.77:0/1058997546' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T09:25:06.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:05 smithi003 ceph-mon[25908]: from='mgr.? 172.21.15.77:0/1058997546' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T09:25:06.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:05 smithi003 ceph-mon[25908]: from='mgr.? 172.21.15.77:0/1058997546' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T09:25:06.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:05 smithi003 ceph-mon[25908]: from='mgr.? 172.21.15.77:0/1058997546' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T09:25:06.314 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:05 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:05.862+0000 7f0d722f6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T09:25:06.314 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:05 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: [17/Sep/2024:09:25:05] ENGINE Bus STARTING 2024-09-17T09:25:06.314 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:05 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: CherryPy Checker: 2024-09-17T09:25:06.314 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:05 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: The Application mounted at '' has an empty config. 2024-09-17T09:25:06.314 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:05 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: 2024-09-17T09:25:06.314 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:05 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: [17/Sep/2024:09:25:05] ENGINE Serving on http://:::9283 2024-09-17T09:25:06.314 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:25:05 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z[32424]: [17/Sep/2024:09:25:05] ENGINE Bus STARTED 2024-09-17T09:25:06.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:05 smithi183 ceph-mon[30372]: Standby manager daemon z restarted 2024-09-17T09:25:06.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:05 smithi183 ceph-mon[30372]: Standby manager daemon z started 2024-09-17T09:25:06.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:05 smithi183 ceph-mon[30372]: from='mgr.? 172.21.15.183:0/1581754822' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-17T09:25:06.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:05 smithi183 ceph-mon[30372]: from='mgr.? 172.21.15.183:0/1581754822' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T09:25:06.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:05 smithi183 ceph-mon[30372]: from='mgr.? 172.21.15.183:0/1581754822' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-17T09:25:06.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:05 smithi183 ceph-mon[30372]: from='mgr.? 172.21.15.183:0/1581754822' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T09:25:06.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:05 smithi183 ceph-mon[30372]: Standby manager daemon x restarted 2024-09-17T09:25:06.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:05 smithi183 ceph-mon[30372]: Standby manager daemon x started 2024-09-17T09:25:06.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:05 smithi183 ceph-mon[30372]: from='mgr.? 172.21.15.77:0/1058997546' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T09:25:06.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:05 smithi183 ceph-mon[30372]: from='mgr.? 172.21.15.77:0/1058997546' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T09:25:06.316 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:05 smithi183 ceph-mon[30372]: from='mgr.? 172.21.15.77:0/1058997546' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T09:25:06.316 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:05 smithi183 ceph-mon[30372]: from='mgr.? 172.21.15.77:0/1058997546' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T09:25:06.533 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:06.317+0000 7f9d82fb2040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T09:25:06.533 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: [17/Sep/2024:09:25:06] ENGINE Bus STARTING 2024-09-17T09:25:06.533 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: CherryPy Checker: 2024-09-17T09:25:06.533 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: The Application mounted at '' has an empty config. 2024-09-17T09:25:06.533 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:06.811 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: [17/Sep/2024:09:25:06] ENGINE Serving on http://:::9283 2024-09-17T09:25:06.812 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: [17/Sep/2024:09:25:06] ENGINE Bus STARTED 2024-09-17T09:25:07.073 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: mgrmap e18: y(active, since 2m), standbys: x, z 2024-09-17T09:25:07.073 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: Active manager daemon y restarted 2024-09-17T09:25:07.073 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: Activating manager daemon y 2024-09-17T09:25:07.074 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: osdmap e31: 3 total, 3 up, 3 in 2024-09-17T09:25:07.074 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: mgrmap e19: y(active, starting, since 0.0106504s), standbys: x, z 2024-09-17T09:25:07.074 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:25:07.074 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:25:07.074 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:25:07.074 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T09:25:07.074 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T09:25:07.074 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-17T09:25:07.074 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:25:07.074 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:25:07.074 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:25:07.074 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T09:25:07.074 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T09:25:07.074 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T09:25:07.075 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: Manager daemon y is now available 2024-09-17T09:25:07.075 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:07.075 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:07.075 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:25:07.075 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T09:25:07.075 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:06 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T09:25:07.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: mgrmap e18: y(active, since 2m), standbys: x, z 2024-09-17T09:25:07.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: Active manager daemon y restarted 2024-09-17T09:25:07.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: Activating manager daemon y 2024-09-17T09:25:07.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: osdmap e31: 3 total, 3 up, 3 in 2024-09-17T09:25:07.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: mgrmap e19: y(active, starting, since 0.0106504s), standbys: x, z 2024-09-17T09:25:07.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:25:07.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:25:07.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:25:07.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T09:25:07.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T09:25:07.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-17T09:25:07.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:25:07.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:25:07.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:25:07.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T09:25:07.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T09:25:07.253 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T09:25:07.254 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: Manager daemon y is now available 2024-09-17T09:25:07.254 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:07.254 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:07.254 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:25:07.254 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T09:25:07.254 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:06 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T09:25:07.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: mgrmap e18: y(active, since 2m), standbys: x, z 2024-09-17T09:25:07.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: Active manager daemon y restarted 2024-09-17T09:25:07.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: Activating manager daemon y 2024-09-17T09:25:07.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: osdmap e31: 3 total, 3 up, 3 in 2024-09-17T09:25:07.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: mgrmap e19: y(active, starting, since 0.0106504s), standbys: x, z 2024-09-17T09:25:07.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:25:07.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:25:07.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:25:07.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T09:25:07.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T09:25:07.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-17T09:25:07.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:25:07.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:25:07.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:25:07.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T09:25:07.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T09:25:07.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T09:25:07.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: Manager daemon y is now available 2024-09-17T09:25:07.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:07.316 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:07.316 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:25:07.316 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T09:25:07.316 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:06 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T09:25:08.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:08 smithi077 ceph-mon[30659]: mgrmap e20: y(active, since 1.01385s), standbys: x, z 2024-09-17T09:25:08.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:08 smithi077 ceph-mon[30659]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:08.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:08 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:08.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:08 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:08.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-mon[25908]: mgrmap e20: y(active, since 1.01385s), standbys: x, z 2024-09-17T09:25:08.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-mon[25908]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:08.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:08.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:08.813 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:08 smithi183 ceph-mon[30372]: mgrmap e20: y(active, since 1.01385s), standbys: x, z 2024-09-17T09:25:08.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:08 smithi183 ceph-mon[30372]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:08.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:08 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:08.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:08 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:08.998 INFO:teuthology.orchestra.run.smithi003.stdout:34359738387 2024-09-17T09:25:08.999 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-17T09:25:09.228 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:08.977+0000 7f9cd6c58640 -1 log_channel(cephadm) log [ERR] : [17/Sep/2024:09:25:08] ENGINE Error in HTTPServer.serve 2024-09-17T09:25:09.228 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: Traceback (most recent call last): 2024-09-17T09:25:09.228 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T09:25:09.228 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: self._connections.run(self.expiration_interval) 2024-09-17T09:25:09.228 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T09:25:09.228 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: self._run(expiration_interval) 2024-09-17T09:25:09.229 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T09:25:09.229 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T09:25:09.229 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T09:25:09.229 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T09:25:09.229 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T09:25:09.229 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: s = self.context.wrap_socket( 2024-09-17T09:25:09.229 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T09:25:09.229 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: return self.sslsocket_class._create( 2024-09-17T09:25:09.229 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T09:25:09.229 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: self.do_handshake() 2024-09-17T09:25:09.229 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T09:25:09.229 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: self._sslobj.do_handshake() 2024-09-17T09:25:09.229 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T09:25:09.230 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:08 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: 2024-09-17T09:25:09.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:09 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:09 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:09 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:09 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:09 smithi003 ceph-mon[25908]: [17/Sep/2024:09:25:08] ENGINE Bus STARTING 2024-09-17T09:25:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:09 smithi003 ceph-mon[25908]: [17/Sep/2024:09:25:08] ENGINE Serving on http://172.21.15.3:8765 2024-09-17T09:25:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:09 smithi003 ceph-mon[25908]: mgrmap e21: y(active, since 2s), standbys: x, z 2024-09-17T09:25:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:09 smithi003 ceph-mon[25908]: [17/Sep/2024:09:25:08] ENGINE Serving on https://172.21.15.3:7150 2024-09-17T09:25:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:09 smithi003 ceph-mon[25908]: [17/Sep/2024:09:25:08] ENGINE Bus STARTED 2024-09-17T09:25:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:09 smithi003 ceph-mon[25908]: [17/Sep/2024:09:25:08] ENGINE Error in HTTPServer.serve 2024-09-17T09:25:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout: Traceback (most recent call last): 2024-09-17T09:25:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T09:25:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._connections.run(self.expiration_interval) 2024-09-17T09:25:09.509 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T09:25:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._run(expiration_interval) 2024-09-17T09:25:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T09:25:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T09:25:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T09:25:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T09:25:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T09:25:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout: s = self.context.wrap_socket( 2024-09-17T09:25:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T09:25:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout: return self.sslsocket_class._create( 2024-09-17T09:25:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T09:25:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout: self.do_handshake() 2024-09-17T09:25:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T09:25:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._sslobj.do_handshake() 2024-09-17T09:25:09.510 INFO:journalctl@ceph.mon.a.smithi003.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T09:25:09.511 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:09 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:09.511 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:09 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:09.511 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:09 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:25:09.511 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:09 smithi003 ceph-mon[25908]: Adjusting osd_memory_target on smithi077 to 16064M 2024-09-17T09:25:09.511 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:09 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:09.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:09 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:09.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:09 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:09.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:09 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:09.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:09 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:09.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:09 smithi077 ceph-mon[30659]: [17/Sep/2024:09:25:08] ENGINE Bus STARTING 2024-09-17T09:25:09.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:09 smithi077 ceph-mon[30659]: [17/Sep/2024:09:25:08] ENGINE Serving on http://172.21.15.3:8765 2024-09-17T09:25:09.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:09 smithi077 ceph-mon[30659]: mgrmap e21: y(active, since 2s), standbys: x, z 2024-09-17T09:25:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:09 smithi077 ceph-mon[30659]: [17/Sep/2024:09:25:08] ENGINE Serving on https://172.21.15.3:7150 2024-09-17T09:25:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:09 smithi077 ceph-mon[30659]: [17/Sep/2024:09:25:08] ENGINE Bus STARTED 2024-09-17T09:25:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:09 smithi077 ceph-mon[30659]: [17/Sep/2024:09:25:08] ENGINE Error in HTTPServer.serve 2024-09-17T09:25:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout: Traceback (most recent call last): 2024-09-17T09:25:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T09:25:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout: self._connections.run(self.expiration_interval) 2024-09-17T09:25:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T09:25:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout: self._run(expiration_interval) 2024-09-17T09:25:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T09:25:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T09:25:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T09:25:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T09:25:09.753 INFO:journalctl@ceph.mon.b.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T09:25:09.754 INFO:journalctl@ceph.mon.b.smithi077.stdout: s = self.context.wrap_socket( 2024-09-17T09:25:09.754 INFO:journalctl@ceph.mon.b.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T09:25:09.754 INFO:journalctl@ceph.mon.b.smithi077.stdout: return self.sslsocket_class._create( 2024-09-17T09:25:09.754 INFO:journalctl@ceph.mon.b.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T09:25:09.754 INFO:journalctl@ceph.mon.b.smithi077.stdout: self.do_handshake() 2024-09-17T09:25:09.754 INFO:journalctl@ceph.mon.b.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T09:25:09.754 INFO:journalctl@ceph.mon.b.smithi077.stdout: self._sslobj.do_handshake() 2024-09-17T09:25:09.754 INFO:journalctl@ceph.mon.b.smithi077.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T09:25:09.754 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:09 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:09.754 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:09 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:09.754 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:09 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:25:09.754 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:09 smithi077 ceph-mon[30659]: Adjusting osd_memory_target on smithi077 to 16064M 2024-09-17T09:25:09.754 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:09 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:09.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:09 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:09.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:09 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:09.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:09 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:09.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:09 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:09.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:09 smithi183 ceph-mon[30372]: [17/Sep/2024:09:25:08] ENGINE Bus STARTING 2024-09-17T09:25:09.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:09 smithi183 ceph-mon[30372]: [17/Sep/2024:09:25:08] ENGINE Serving on http://172.21.15.3:8765 2024-09-17T09:25:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:09 smithi183 ceph-mon[30372]: mgrmap e21: y(active, since 2s), standbys: x, z 2024-09-17T09:25:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:09 smithi183 ceph-mon[30372]: [17/Sep/2024:09:25:08] ENGINE Serving on https://172.21.15.3:7150 2024-09-17T09:25:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:09 smithi183 ceph-mon[30372]: [17/Sep/2024:09:25:08] ENGINE Bus STARTED 2024-09-17T09:25:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:09 smithi183 ceph-mon[30372]: [17/Sep/2024:09:25:08] ENGINE Error in HTTPServer.serve 2024-09-17T09:25:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout: Traceback (most recent call last): 2024-09-17T09:25:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T09:25:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout: self._connections.run(self.expiration_interval) 2024-09-17T09:25:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T09:25:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout: self._run(expiration_interval) 2024-09-17T09:25:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T09:25:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T09:25:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T09:25:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T09:25:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T09:25:09.815 INFO:journalctl@ceph.mon.c.smithi183.stdout: s = self.context.wrap_socket( 2024-09-17T09:25:09.816 INFO:journalctl@ceph.mon.c.smithi183.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T09:25:09.816 INFO:journalctl@ceph.mon.c.smithi183.stdout: return self.sslsocket_class._create( 2024-09-17T09:25:09.816 INFO:journalctl@ceph.mon.c.smithi183.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T09:25:09.816 INFO:journalctl@ceph.mon.c.smithi183.stdout: self.do_handshake() 2024-09-17T09:25:09.816 INFO:journalctl@ceph.mon.c.smithi183.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T09:25:09.816 INFO:journalctl@ceph.mon.c.smithi183.stdout: self._sslobj.do_handshake() 2024-09-17T09:25:09.816 INFO:journalctl@ceph.mon.c.smithi183.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T09:25:09.816 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:09 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:09.816 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:09 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:09.816 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:09 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:25:09.816 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:09 smithi183 ceph-mon[30372]: Adjusting osd_memory_target on smithi077 to 16064M 2024-09-17T09:25:09.816 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:09 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:09.905 INFO:teuthology.orchestra.run.smithi003.stdout:77309411339 2024-09-17T09:25:09.906 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-17T09:25:10.325 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:25:11.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:11 smithi003 ceph-mon[25908]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:11.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:11 smithi003 ceph-mon[25908]: mgrmap e22: y(active, since 4s), standbys: x, z 2024-09-17T09:25:11.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:11 smithi077 ceph-mon[30659]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:11.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:11 smithi077 ceph-mon[30659]: mgrmap e22: y(active, since 4s), standbys: x, z 2024-09-17T09:25:11.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:11 smithi183 ceph-mon[30372]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:11.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:11 smithi183 ceph-mon[30372]: mgrmap e22: y(active, since 4s), standbys: x, z 2024-09-17T09:25:12.010 INFO:teuthology.orchestra.run.smithi003.stdout:55834574864 2024-09-17T09:25:12.010 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-17T09:25:13.507 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:25:13.507 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:25:13.633 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:13 smithi003 ceph-mon[25908]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 41 KiB/s rd, 0 B/s wr, 23 op/s 2024-09-17T09:25:13.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:13 smithi077 ceph-mon[30659]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 41 KiB/s rd, 0 B/s wr, 23 op/s 2024-09-17T09:25:13.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:13 smithi183 ceph-mon[30372]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 41 KiB/s rd, 0 B/s wr, 23 op/s 2024-09-17T09:25:13.916 INFO:teuthology.orchestra.run.smithi003.stdout:34359738388 2024-09-17T09:25:13.916 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:13 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: ::ffff:172.21.15.77 - - [17/Sep/2024:09:25:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T09:25:14.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:14 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/1498403348' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-17T09:25:14.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:14 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/1498403348' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-17T09:25:14.800 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738387 got 34359738388 for osd.0 2024-09-17T09:25:14.801 DEBUG:teuthology.parallel:result is None 2024-09-17T09:25:14.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:14 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/1498403348' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-17T09:25:15.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:15 smithi077 ceph-mon[30659]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-17T09:25:15.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:15 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:15.752 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:15 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:15.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:15 smithi003 ceph-mon[25908]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-17T09:25:15.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:15 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:15.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:15 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:15.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:15 smithi183 ceph-mon[30372]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-17T09:25:15.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:15 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:15.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:15 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:25:17.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: Adjusting osd_memory_target on smithi003 to 16068M 2024-09-17T09:25:17.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:17.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:17.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: Updating smithi003:/etc/ceph/ceph.conf 2024-09-17T09:25:17.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: Updating smithi077:/etc/ceph/ceph.conf 2024-09-17T09:25:17.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-17T09:25:17.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: Updating smithi003:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:25:17.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: Updating smithi077:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:25:17.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: Updating smithi183:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:25:17.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:25:17.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:25:17.004 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:25:17.004 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.004 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.004 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.004 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.004 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.004 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.004 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:16 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.006 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.006 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.006 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:25:17.006 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: Adjusting osd_memory_target on smithi003 to 16068M 2024-09-17T09:25:17.006 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.006 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:17.006 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:17.006 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: Updating smithi003:/etc/ceph/ceph.conf 2024-09-17T09:25:17.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: Updating smithi077:/etc/ceph/ceph.conf 2024-09-17T09:25:17.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-17T09:25:17.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: Updating smithi003:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:25:17.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: Updating smithi077:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:25:17.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: Updating smithi183:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:25:17.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:25:17.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:25:17.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:25:17.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.007 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:16 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:25:17.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: Adjusting osd_memory_target on smithi003 to 16068M 2024-09-17T09:25:17.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:17.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:17.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: Updating smithi003:/etc/ceph/ceph.conf 2024-09-17T09:25:17.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: Updating smithi077:/etc/ceph/ceph.conf 2024-09-17T09:25:17.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-17T09:25:17.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: Updating smithi003:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:25:17.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: Updating smithi077:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:25:17.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: Updating smithi183:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:25:17.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:25:17.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:25:17.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:25:17.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.066 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.066 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.066 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.066 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:16 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:17.639 INFO:teuthology.orchestra.run.smithi003.stdout:77309411340 2024-09-17T09:25:17.712 INFO:teuthology.orchestra.run.smithi003.stdout:55834574865 2024-09-17T09:25:17.853 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:17 smithi003 ceph-mon[25908]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-17T09:25:17.853 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:17 smithi003 ceph-mon[25908]: Updating smithi077:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:25:17.853 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:17 smithi003 ceph-mon[25908]: Updating smithi183:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:25:17.853 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:17 smithi003 ceph-mon[25908]: Updating smithi003:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:25:17.853 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:17 smithi003 ceph-mon[25908]: Deploying daemon alertmanager.a on smithi003 2024-09-17T09:25:17.853 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:17 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/1592799359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-17T09:25:17.853 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:17 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/1933609530' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-17T09:25:18.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:17 smithi077 ceph-mon[30659]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-17T09:25:18.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:17 smithi077 ceph-mon[30659]: Updating smithi077:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:25:18.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:17 smithi077 ceph-mon[30659]: Updating smithi183:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:25:18.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:17 smithi077 ceph-mon[30659]: Updating smithi003:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:25:18.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:17 smithi077 ceph-mon[30659]: Deploying daemon alertmanager.a on smithi003 2024-09-17T09:25:18.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:17 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/1592799359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-17T09:25:18.003 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:17 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/1933609530' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-17T09:25:18.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:17 smithi183 ceph-mon[30372]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-17T09:25:18.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:17 smithi183 ceph-mon[30372]: Updating smithi077:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:25:18.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:17 smithi183 ceph-mon[30372]: Updating smithi183:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:25:18.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:17 smithi183 ceph-mon[30372]: Updating smithi003:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:25:18.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:17 smithi183 ceph-mon[30372]: Deploying daemon alertmanager.a on smithi003 2024-09-17T09:25:18.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:17 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/1592799359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-17T09:25:18.065 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:17 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/1933609530' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-17T09:25:18.398 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574864 got 55834574865 for osd.1 2024-09-17T09:25:18.399 DEBUG:teuthology.parallel:result is None 2024-09-17T09:25:18.842 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411339 got 77309411340 for osd.2 2024-09-17T09:25:18.842 DEBUG:teuthology.parallel:result is None 2024-09-17T09:25:18.842 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-17T09:25:18.843 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T09:25:19.059 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:25:19.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:19 smithi003 ceph-mon[25908]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T09:25:20.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:19 smithi077 ceph-mon[30659]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T09:25:20.063 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:19 smithi183 ceph-mon[30372]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T09:25:21.340 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:25:21.341 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-17T09:25:22.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:21 smithi077 ceph-mon[30659]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-17T09:25:22.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:21 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:25:22.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:21 smithi003 ceph-mon[25908]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-17T09:25:22.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:21 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:25:22.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:21 smithi183 ceph-mon[30372]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-17T09:25:22.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:21 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:25:22.882 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-17T09:25:20.338755+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595838,"num_objects":193,"num_object_clones":0,"num_object_copies":579,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":193,"num_whiteouts":0,"num_read":962,"num_read_kb":809,"num_write":691,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":669,"ondisk_log_size":669,"up":387,"acting":387,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":387,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":281223168,"kb_used":96400,"kb_used_data":3056,"kb_used_omap":4,"kb_used_meta":93307,"kb_avail":281126768,"statfs":{"total":287972524032,"available":287873810432,"internally_reserved":0,"allocated":3129344,"data_stored":2099262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4782,"internal_metadata":95546706},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":206,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002393"},"pg_stats":[{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356988+0000","last_change":"2024-09-17T09:24:41.288691+0000","last_active":"2024-09-17T09:24:48.356988+0000","last_peered":"2024-09-17T09:24:48.356988+0000","last_clean":"2024-09-17T09:24:48.356988+0000","last_became_active":"2024-09-17T09:24:41.288367+0000","last_became_peered":"2024-09-17T09:24:41.288367+0000","last_unstale":"2024-09-17T09:24:48.356988+0000","last_undegraded":"2024-09-17T09:24:48.356988+0000","last_fullsized":"2024-09-17T09:24:48.356988+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:25.055167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'10","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.665875+0000","last_change":"2024-09-17T09:24:43.340996+0000","last_active":"2024-09-17T09:24:48.665875+0000","last_peered":"2024-09-17T09:24:48.665875+0000","last_clean":"2024-09-17T09:24:48.665875+0000","last_became_active":"2024-09-17T09:24:43.340648+0000","last_became_peered":"2024-09-17T09:24:43.340648+0000","last_unstale":"2024-09-17T09:24:48.665875+0000","last_undegraded":"2024-09-17T09:24:48.665875+0000","last_fullsized":"2024-09-17T09:24:48.665875+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:29:24.836425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356099+0000","last_change":"2024-09-17T09:24:45.356241+0000","last_active":"2024-09-17T09:24:48.356099+0000","last_peered":"2024-09-17T09:24:48.356099+0000","last_clean":"2024-09-17T09:24:48.356099+0000","last_became_active":"2024-09-17T09:24:45.355977+0000","last_became_peered":"2024-09-17T09:24:45.355977+0000","last_unstale":"2024-09-17T09:24:48.356099+0000","last_undegraded":"2024-09-17T09:24:48.356099+0000","last_fullsized":"2024-09-17T09:24:48.356099+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:03:19.477277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519072+0000","last_change":"2024-09-17T09:24:47.366128+0000","last_active":"2024-09-17T09:24:49.519072+0000","last_peered":"2024-09-17T09:24:49.519072+0000","last_clean":"2024-09-17T09:24:49.519072+0000","last_became_active":"2024-09-17T09:24:47.365112+0000","last_became_peered":"2024-09-17T09:24:47.365112+0000","last_unstale":"2024-09-17T09:24:49.519072+0000","last_undegraded":"2024-09-17T09:24:49.519072+0000","last_fullsized":"2024-09-17T09:24:49.519072+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:29:15.425415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357021+0000","last_change":"2024-09-17T09:24:41.289409+0000","last_active":"2024-09-17T09:24:48.357021+0000","last_peered":"2024-09-17T09:24:48.357021+0000","last_clean":"2024-09-17T09:24:48.357021+0000","last_became_active":"2024-09-17T09:24:41.289152+0000","last_became_peered":"2024-09-17T09:24:41.289152+0000","last_unstale":"2024-09-17T09:24:48.357021+0000","last_undegraded":"2024-09-17T09:24:48.357021+0000","last_fullsized":"2024-09-17T09:24:48.357021+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:41:08.494827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"30'11","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.672303+0000","last_change":"2024-09-17T09:24:43.342559+0000","last_active":"2024-09-17T09:24:48.672303+0000","last_peered":"2024-09-17T09:24:48.672303+0000","last_clean":"2024-09-17T09:24:48.672303+0000","last_became_active":"2024-09-17T09:24:43.342014+0000","last_became_peered":"2024-09-17T09:24:43.342014+0000","last_unstale":"2024-09-17T09:24:48.672303+0000","last_undegraded":"2024-09-17T09:24:48.672303+0000","last_fullsized":"2024-09-17T09:24:48.672303+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:05:26.826249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.358699+0000","last_change":"2024-09-17T09:24:45.361092+0000","last_active":"2024-09-17T09:24:48.358699+0000","last_peered":"2024-09-17T09:24:48.358699+0000","last_clean":"2024-09-17T09:24:48.358699+0000","last_became_active":"2024-09-17T09:24:45.360645+0000","last_became_peered":"2024-09-17T09:24:45.360645+0000","last_unstale":"2024-09-17T09:24:48.358699+0000","last_undegraded":"2024-09-17T09:24:48.358699+0000","last_fullsized":"2024-09-17T09:24:48.358699+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:24.528070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355640+0000","last_change":"2024-09-17T09:24:47.368536+0000","last_active":"2024-09-17T09:24:48.355640+0000","last_peered":"2024-09-17T09:24:48.355640+0000","last_clean":"2024-09-17T09:24:48.355640+0000","last_became_active":"2024-09-17T09:24:47.368224+0000","last_became_peered":"2024-09-17T09:24:47.368224+0000","last_unstale":"2024-09-17T09:24:48.355640+0000","last_undegraded":"2024-09-17T09:24:48.355640+0000","last_fullsized":"2024-09-17T09:24:48.355640+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:26:56.921083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357675+0000","last_change":"2024-09-17T09:24:41.292706+0000","last_active":"2024-09-17T09:24:48.357675+0000","last_peered":"2024-09-17T09:24:48.357675+0000","last_clean":"2024-09-17T09:24:48.357675+0000","last_became_active":"2024-09-17T09:24:41.291355+0000","last_became_peered":"2024-09-17T09:24:41.291355+0000","last_unstale":"2024-09-17T09:24:48.357675+0000","last_undegraded":"2024-09-17T09:24:48.357675+0000","last_fullsized":"2024-09-17T09:24:48.357675+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:19:44.827354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"30'15","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.614974+0000","last_change":"2024-09-17T09:24:43.344325+0000","last_active":"2024-09-17T09:24:48.614974+0000","last_peered":"2024-09-17T09:24:48.614974+0000","last_clean":"2024-09-17T09:24:48.614974+0000","last_became_active":"2024-09-17T09:24:43.344005+0000","last_became_peered":"2024-09-17T09:24:43.344005+0000","last_unstale":"2024-09-17T09:24:48.614974+0000","last_undegraded":"2024-09-17T09:24:48.614974+0000","last_fullsized":"2024-09-17T09:24:48.614974+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:52:17.687949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.360323+0000","last_change":"2024-09-17T09:24:45.358148+0000","last_active":"2024-09-17T09:24:48.360323+0000","last_peered":"2024-09-17T09:24:48.360323+0000","last_clean":"2024-09-17T09:24:48.360323+0000","last_became_active":"2024-09-17T09:24:45.356561+0000","last_became_peered":"2024-09-17T09:24:45.356561+0000","last_unstale":"2024-09-17T09:24:48.360323+0000","last_undegraded":"2024-09-17T09:24:48.360323+0000","last_fullsized":"2024-09-17T09:24:48.360323+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:33.833413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356161+0000","last_change":"2024-09-17T09:24:47.362199+0000","last_active":"2024-09-17T09:24:48.356161+0000","last_peered":"2024-09-17T09:24:48.356161+0000","last_clean":"2024-09-17T09:24:48.356161+0000","last_became_active":"2024-09-17T09:24:47.361927+0000","last_became_peered":"2024-09-17T09:24:47.361927+0000","last_unstale":"2024-09-17T09:24:48.356161+0000","last_undegraded":"2024-09-17T09:24:48.356161+0000","last_fullsized":"2024-09-17T09:24:48.356161+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:18.507166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356935+0000","last_change":"2024-09-17T09:24:41.291521+0000","last_active":"2024-09-17T09:24:48.356935+0000","last_peered":"2024-09-17T09:24:48.356935+0000","last_clean":"2024-09-17T09:24:48.356935+0000","last_became_active":"2024-09-17T09:24:41.291099+0000","last_became_peered":"2024-09-17T09:24:41.291099+0000","last_unstale":"2024-09-17T09:24:48.356935+0000","last_undegraded":"2024-09-17T09:24:48.356935+0000","last_fullsized":"2024-09-17T09:24:48.356935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:58:53.610252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"30'12","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.678612+0000","last_change":"2024-09-17T09:24:43.340649+0000","last_active":"2024-09-17T09:24:48.678612+0000","last_peered":"2024-09-17T09:24:48.678612+0000","last_clean":"2024-09-17T09:24:48.678612+0000","last_became_active":"2024-09-17T09:24:43.340387+0000","last_became_peered":"2024-09-17T09:24:43.340387+0000","last_unstale":"2024-09-17T09:24:48.678612+0000","last_undegraded":"2024-09-17T09:24:48.678612+0000","last_fullsized":"2024-09-17T09:24:48.678612+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:44:44.539535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.359225+0000","last_change":"2024-09-17T09:24:45.361135+0000","last_active":"2024-09-17T09:24:48.359225+0000","last_peered":"2024-09-17T09:24:48.359225+0000","last_clean":"2024-09-17T09:24:48.359225+0000","last_became_active":"2024-09-17T09:24:45.360800+0000","last_became_peered":"2024-09-17T09:24:45.360800+0000","last_unstale":"2024-09-17T09:24:48.359225+0000","last_undegraded":"2024-09-17T09:24:48.359225+0000","last_fullsized":"2024-09-17T09:24:48.359225+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:28:58.469674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519233+0000","last_change":"2024-09-17T09:24:47.365651+0000","last_active":"2024-09-17T09:24:49.519233+0000","last_peered":"2024-09-17T09:24:49.519233+0000","last_clean":"2024-09-17T09:24:49.519233+0000","last_became_active":"2024-09-17T09:24:47.364640+0000","last_became_peered":"2024-09-17T09:24:47.364640+0000","last_unstale":"2024-09-17T09:24:49.519233+0000","last_undegraded":"2024-09-17T09:24:49.519233+0000","last_fullsized":"2024-09-17T09:24:49.519233+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:55:16.538127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"30'19","reported_seq":47,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.682598+0000","last_change":"2024-09-17T09:24:43.343505+0000","last_active":"2024-09-17T09:24:48.682598+0000","last_peered":"2024-09-17T09:24:48.682598+0000","last_clean":"2024-09-17T09:24:48.682598+0000","last_became_active":"2024-09-17T09:24:43.342721+0000","last_became_peered":"2024-09-17T09:24:43.342721+0000","last_unstale":"2024-09-17T09:24:48.682598+0000","last_undegraded":"2024-09-17T09:24:48.682598+0000","last_fullsized":"2024-09-17T09:24:48.682598+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:55.795368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356410+0000","last_change":"2024-09-17T09:24:41.291806+0000","last_active":"2024-09-17T09:24:48.356410+0000","last_peered":"2024-09-17T09:24:48.356410+0000","last_clean":"2024-09-17T09:24:48.356410+0000","last_became_active":"2024-09-17T09:24:41.291541+0000","last_became_peered":"2024-09-17T09:24:41.291541+0000","last_unstale":"2024-09-17T09:24:48.356410+0000","last_undegraded":"2024-09-17T09:24:48.356410+0000","last_fullsized":"2024-09-17T09:24:48.356410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:35:47.673467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"30'8","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T09:25:09.573505+0000","last_change":"2024-09-17T09:24:45.358788+0000","last_active":"2024-09-17T09:25:09.573505+0000","last_peered":"2024-09-17T09:25:09.573505+0000","last_clean":"2024-09-17T09:25:09.573505+0000","last_became_active":"2024-09-17T09:24:45.357990+0000","last_became_peered":"2024-09-17T09:24:45.357990+0000","last_unstale":"2024-09-17T09:25:09.573505+0000","last_undegraded":"2024-09-17T09:25:09.573505+0000","last_fullsized":"2024-09-17T09:25:09.573505+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:50.747097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356001+0000","last_change":"2024-09-17T09:24:47.365848+0000","last_active":"2024-09-17T09:24:48.356001+0000","last_peered":"2024-09-17T09:24:48.356001+0000","last_clean":"2024-09-17T09:24:48.356001+0000","last_became_active":"2024-09-17T09:24:47.365569+0000","last_became_peered":"2024-09-17T09:24:47.365569+0000","last_unstale":"2024-09-17T09:24:48.356001+0000","last_undegraded":"2024-09-17T09:24:48.356001+0000","last_fullsized":"2024-09-17T09:24:48.356001+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:17:10.499880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.645311+0000","last_change":"2024-09-17T09:24:43.339916+0000","last_active":"2024-09-17T09:24:48.645311+0000","last_peered":"2024-09-17T09:24:48.645311+0000","last_clean":"2024-09-17T09:24:48.645311+0000","last_became_active":"2024-09-17T09:24:43.339578+0000","last_became_peered":"2024-09-17T09:24:43.339578+0000","last_unstale":"2024-09-17T09:24:48.645311+0000","last_undegraded":"2024-09-17T09:24:48.645311+0000","last_fullsized":"2024-09-17T09:24:48.645311+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:08:30.859802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356140+0000","last_change":"2024-09-17T09:24:41.292820+0000","last_active":"2024-09-17T09:24:48.356140+0000","last_peered":"2024-09-17T09:24:48.356140+0000","last_clean":"2024-09-17T09:24:48.356140+0000","last_became_active":"2024-09-17T09:24:41.292241+0000","last_became_peered":"2024-09-17T09:24:41.292241+0000","last_unstale":"2024-09-17T09:24:48.356140+0000","last_undegraded":"2024-09-17T09:24:48.356140+0000","last_fullsized":"2024-09-17T09:24:48.356140+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:03.940823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.360291+0000","last_change":"2024-09-17T09:24:45.360696+0000","last_active":"2024-09-17T09:24:48.360291+0000","last_peered":"2024-09-17T09:24:48.360291+0000","last_clean":"2024-09-17T09:24:48.360291+0000","last_became_active":"2024-09-17T09:24:45.360374+0000","last_became_peered":"2024-09-17T09:24:45.360374+0000","last_unstale":"2024-09-17T09:24:48.360291+0000","last_undegraded":"2024-09-17T09:24:48.360291+0000","last_fullsized":"2024-09-17T09:24:48.360291+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:01.952661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.358185+0000","last_change":"2024-09-17T09:24:47.363252+0000","last_active":"2024-09-17T09:24:48.358185+0000","last_peered":"2024-09-17T09:24:48.358185+0000","last_clean":"2024-09-17T09:24:48.358185+0000","last_became_active":"2024-09-17T09:24:47.362918+0000","last_became_peered":"2024-09-17T09:24:47.362918+0000","last_unstale":"2024-09-17T09:24:48.358185+0000","last_undegraded":"2024-09-17T09:24:48.358185+0000","last_fullsized":"2024-09-17T09:24:48.358185+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:37:33.329282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"30'15","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.620920+0000","last_change":"2024-09-17T09:24:43.341292+0000","last_active":"2024-09-17T09:24:48.620920+0000","last_peered":"2024-09-17T09:24:48.620920+0000","last_clean":"2024-09-17T09:24:48.620920+0000","last_became_active":"2024-09-17T09:24:43.341004+0000","last_became_peered":"2024-09-17T09:24:43.341004+0000","last_unstale":"2024-09-17T09:24:48.620920+0000","last_undegraded":"2024-09-17T09:24:48.620920+0000","last_fullsized":"2024-09-17T09:24:48.620920+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:41.425131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355219+0000","last_change":"2024-09-17T09:24:41.289622+0000","last_active":"2024-09-17T09:24:48.355219+0000","last_peered":"2024-09-17T09:24:48.355219+0000","last_clean":"2024-09-17T09:24:48.355219+0000","last_became_active":"2024-09-17T09:24:41.289290+0000","last_became_peered":"2024-09-17T09:24:41.289290+0000","last_unstale":"2024-09-17T09:24:48.355219+0000","last_undegraded":"2024-09-17T09:24:48.355219+0000","last_fullsized":"2024-09-17T09:24:48.355219+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:48:58.205128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357391+0000","last_change":"2024-09-17T09:24:45.356706+0000","last_active":"2024-09-17T09:24:48.357391+0000","last_peered":"2024-09-17T09:24:48.357391+0000","last_clean":"2024-09-17T09:24:48.357391+0000","last_became_active":"2024-09-17T09:24:45.356282+0000","last_became_peered":"2024-09-17T09:24:45.356282+0000","last_unstale":"2024-09-17T09:24:48.357391+0000","last_undegraded":"2024-09-17T09:24:48.357391+0000","last_fullsized":"2024-09-17T09:24:48.357391+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:03.128580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.358527+0000","last_change":"2024-09-17T09:24:47.363118+0000","last_active":"2024-09-17T09:24:48.358527+0000","last_peered":"2024-09-17T09:24:48.358527+0000","last_clean":"2024-09-17T09:24:48.358527+0000","last_became_active":"2024-09-17T09:24:47.362848+0000","last_became_peered":"2024-09-17T09:24:47.362848+0000","last_unstale":"2024-09-17T09:24:48.358527+0000","last_undegraded":"2024-09-17T09:24:48.358527+0000","last_fullsized":"2024-09-17T09:24:48.358527+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:28.334802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"30'12","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.506426+0000","last_change":"2024-09-17T09:24:43.341068+0000","last_active":"2024-09-17T09:24:48.506426+0000","last_peered":"2024-09-17T09:24:48.506426+0000","last_clean":"2024-09-17T09:24:48.506426+0000","last_became_active":"2024-09-17T09:24:43.340760+0000","last_became_peered":"2024-09-17T09:24:43.340760+0000","last_unstale":"2024-09-17T09:24:48.506426+0000","last_undegraded":"2024-09-17T09:24:48.506426+0000","last_fullsized":"2024-09-17T09:24:48.506426+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:16:40.473848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.358725+0000","last_change":"2024-09-17T09:24:41.290572+0000","last_active":"2024-09-17T09:24:48.358725+0000","last_peered":"2024-09-17T09:24:48.358725+0000","last_clean":"2024-09-17T09:24:48.358725+0000","last_became_active":"2024-09-17T09:24:41.290167+0000","last_became_peered":"2024-09-17T09:24:41.290167+0000","last_unstale":"2024-09-17T09:24:48.358725+0000","last_undegraded":"2024-09-17T09:24:48.358725+0000","last_fullsized":"2024-09-17T09:24:48.358725+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:30.727431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"30'8","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T09:25:09.573404+0000","last_change":"2024-09-17T09:24:45.360811+0000","last_active":"2024-09-17T09:25:09.573404+0000","last_peered":"2024-09-17T09:25:09.573404+0000","last_clean":"2024-09-17T09:25:09.573404+0000","last_became_active":"2024-09-17T09:24:45.360518+0000","last_became_peered":"2024-09-17T09:24:45.360518+0000","last_unstale":"2024-09-17T09:25:09.573404+0000","last_undegraded":"2024-09-17T09:25:09.573404+0000","last_fullsized":"2024-09-17T09:25:09.573404+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:00:46.546422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519389+0000","last_change":"2024-09-17T09:24:47.366816+0000","last_active":"2024-09-17T09:24:49.519389+0000","last_peered":"2024-09-17T09:24:49.519389+0000","last_clean":"2024-09-17T09:24:49.519389+0000","last_became_active":"2024-09-17T09:24:47.365986+0000","last_became_peered":"2024-09-17T09:24:47.365986+0000","last_unstale":"2024-09-17T09:24:49.519389+0000","last_undegraded":"2024-09-17T09:24:49.519389+0000","last_fullsized":"2024-09-17T09:24:49.519389+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:39.707829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"30'12","reported_seq":34,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.613115+0000","last_change":"2024-09-17T09:24:43.339922+0000","last_active":"2024-09-17T09:24:48.613115+0000","last_peered":"2024-09-17T09:24:48.613115+0000","last_clean":"2024-09-17T09:24:48.613115+0000","last_became_active":"2024-09-17T09:24:43.339504+0000","last_became_peered":"2024-09-17T09:24:43.339504+0000","last_unstale":"2024-09-17T09:24:48.613115+0000","last_undegraded":"2024-09-17T09:24:48.613115+0000","last_fullsized":"2024-09-17T09:24:48.613115+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:56.919668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.360092+0000","last_change":"2024-09-17T09:24:41.286119+0000","last_active":"2024-09-17T09:24:48.360092+0000","last_peered":"2024-09-17T09:24:48.360092+0000","last_clean":"2024-09-17T09:24:48.360092+0000","last_became_active":"2024-09-17T09:24:41.285741+0000","last_became_peered":"2024-09-17T09:24:41.285741+0000","last_unstale":"2024-09-17T09:24:48.360092+0000","last_undegraded":"2024-09-17T09:24:48.360092+0000","last_fullsized":"2024-09-17T09:24:48.360092+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:03:08.366768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356488+0000","last_change":"2024-09-17T09:24:45.357383+0000","last_active":"2024-09-17T09:24:48.356488+0000","last_peered":"2024-09-17T09:24:48.356488+0000","last_clean":"2024-09-17T09:24:48.356488+0000","last_became_active":"2024-09-17T09:24:45.357100+0000","last_became_peered":"2024-09-17T09:24:45.357100+0000","last_unstale":"2024-09-17T09:24:48.356488+0000","last_undegraded":"2024-09-17T09:24:48.356488+0000","last_fullsized":"2024-09-17T09:24:48.356488+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:39:44.780737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355865+0000","last_change":"2024-09-17T09:24:47.363041+0000","last_active":"2024-09-17T09:24:48.355865+0000","last_peered":"2024-09-17T09:24:48.355865+0000","last_clean":"2024-09-17T09:24:48.355865+0000","last_became_active":"2024-09-17T09:24:47.362691+0000","last_became_peered":"2024-09-17T09:24:47.362691+0000","last_unstale":"2024-09-17T09:24:48.355865+0000","last_undegraded":"2024-09-17T09:24:48.355865+0000","last_fullsized":"2024-09-17T09:24:48.355865+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:40:21.814776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"30'13","reported_seq":43,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.603883+0000","last_change":"2024-09-17T09:24:43.341513+0000","last_active":"2024-09-17T09:24:48.603883+0000","last_peered":"2024-09-17T09:24:48.603883+0000","last_clean":"2024-09-17T09:24:48.603883+0000","last_became_active":"2024-09-17T09:24:43.341100+0000","last_became_peered":"2024-09-17T09:24:43.341100+0000","last_unstale":"2024-09-17T09:24:48.603883+0000","last_undegraded":"2024-09-17T09:24:48.603883+0000","last_fullsized":"2024-09-17T09:24:48.603883+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:48:44.287485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356626+0000","last_change":"2024-09-17T09:24:41.286001+0000","last_active":"2024-09-17T09:24:48.356626+0000","last_peered":"2024-09-17T09:24:48.356626+0000","last_clean":"2024-09-17T09:24:48.356626+0000","last_became_active":"2024-09-17T09:24:41.285710+0000","last_became_peered":"2024-09-17T09:24:41.285710+0000","last_unstale":"2024-09-17T09:24:48.356626+0000","last_undegraded":"2024-09-17T09:24:48.356626+0000","last_fullsized":"2024-09-17T09:24:48.356626+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:41.623205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"30'8","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T09:25:09.573030+0000","last_change":"2024-09-17T09:24:45.359134+0000","last_active":"2024-09-17T09:25:09.573030+0000","last_peered":"2024-09-17T09:25:09.573030+0000","last_clean":"2024-09-17T09:25:09.573030+0000","last_became_active":"2024-09-17T09:24:45.358758+0000","last_became_peered":"2024-09-17T09:24:45.358758+0000","last_unstale":"2024-09-17T09:25:09.573030+0000","last_undegraded":"2024-09-17T09:25:09.573030+0000","last_fullsized":"2024-09-17T09:25:09.573030+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:03.918860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519174+0000","last_change":"2024-09-17T09:24:47.364802+0000","last_active":"2024-09-17T09:24:49.519174+0000","last_peered":"2024-09-17T09:24:49.519174+0000","last_clean":"2024-09-17T09:24:49.519174+0000","last_became_active":"2024-09-17T09:24:47.363953+0000","last_became_peered":"2024-09-17T09:24:47.363953+0000","last_unstale":"2024-09-17T09:24:49.519174+0000","last_undegraded":"2024-09-17T09:24:49.519174+0000","last_fullsized":"2024-09-17T09:24:49.519174+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:49:20.868575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"30'28","reported_seq":71,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T09:25:09.573103+0000","last_change":"2024-09-17T09:24:43.342381+0000","last_active":"2024-09-17T09:25:09.573103+0000","last_peered":"2024-09-17T09:25:09.573103+0000","last_clean":"2024-09-17T09:25:09.573103+0000","last_became_active":"2024-09-17T09:24:43.342031+0000","last_became_peered":"2024-09-17T09:24:43.342031+0000","last_unstale":"2024-09-17T09:25:09.573103+0000","last_undegraded":"2024-09-17T09:25:09.573103+0000","last_fullsized":"2024-09-17T09:25:09.573103+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:28:42.373131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357456+0000","last_change":"2024-09-17T09:24:41.292908+0000","last_active":"2024-09-17T09:24:48.357456+0000","last_peered":"2024-09-17T09:24:48.357456+0000","last_clean":"2024-09-17T09:24:48.357456+0000","last_became_active":"2024-09-17T09:24:41.292251+0000","last_became_peered":"2024-09-17T09:24:41.292251+0000","last_unstale":"2024-09-17T09:24:48.357456+0000","last_undegraded":"2024-09-17T09:24:48.357456+0000","last_fullsized":"2024-09-17T09:24:48.357456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:46.175913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"30'8","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T09:25:09.573336+0000","last_change":"2024-09-17T09:24:45.355662+0000","last_active":"2024-09-17T09:25:09.573336+0000","last_peered":"2024-09-17T09:25:09.573336+0000","last_clean":"2024-09-17T09:25:09.573336+0000","last_became_active":"2024-09-17T09:24:45.355400+0000","last_became_peered":"2024-09-17T09:24:45.355400+0000","last_unstale":"2024-09-17T09:25:09.573336+0000","last_undegraded":"2024-09-17T09:25:09.573336+0000","last_fullsized":"2024-09-17T09:25:09.573336+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:35.789402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519210+0000","last_change":"2024-09-17T09:24:47.366360+0000","last_active":"2024-09-17T09:24:49.519210+0000","last_peered":"2024-09-17T09:24:49.519210+0000","last_clean":"2024-09-17T09:24:49.519210+0000","last_became_active":"2024-09-17T09:24:47.366028+0000","last_became_peered":"2024-09-17T09:24:47.366028+0000","last_unstale":"2024-09-17T09:24:49.519210+0000","last_undegraded":"2024-09-17T09:24:49.519210+0000","last_fullsized":"2024-09-17T09:24:49.519210+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:28.403235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"30'14","reported_seq":38,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.659659+0000","last_change":"2024-09-17T09:24:43.344659+0000","last_active":"2024-09-17T09:24:48.659659+0000","last_peered":"2024-09-17T09:24:48.659659+0000","last_clean":"2024-09-17T09:24:48.659659+0000","last_became_active":"2024-09-17T09:24:43.344367+0000","last_became_peered":"2024-09-17T09:24:43.344367+0000","last_unstale":"2024-09-17T09:24:48.659659+0000","last_undegraded":"2024-09-17T09:24:48.659659+0000","last_fullsized":"2024-09-17T09:24:48.659659+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:35.398373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356055+0000","last_change":"2024-09-17T09:24:41.288895+0000","last_active":"2024-09-17T09:24:48.356055+0000","last_peered":"2024-09-17T09:24:48.356055+0000","last_clean":"2024-09-17T09:24:48.356055+0000","last_became_active":"2024-09-17T09:24:41.288403+0000","last_became_peered":"2024-09-17T09:24:41.288403+0000","last_unstale":"2024-09-17T09:24:48.356055+0000","last_undegraded":"2024-09-17T09:24:48.356055+0000","last_fullsized":"2024-09-17T09:24:48.356055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:20:01.176505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357256+0000","last_change":"2024-09-17T09:24:45.357655+0000","last_active":"2024-09-17T09:24:48.357256+0000","last_peered":"2024-09-17T09:24:48.357256+0000","last_clean":"2024-09-17T09:24:48.357256+0000","last_became_active":"2024-09-17T09:24:45.357388+0000","last_became_peered":"2024-09-17T09:24:45.357388+0000","last_unstale":"2024-09-17T09:24:48.357256+0000","last_undegraded":"2024-09-17T09:24:48.357256+0000","last_fullsized":"2024-09-17T09:24:48.357256+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:25:05.989337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355502+0000","last_change":"2024-09-17T09:24:47.366854+0000","last_active":"2024-09-17T09:24:48.355502+0000","last_peered":"2024-09-17T09:24:48.355502+0000","last_clean":"2024-09-17T09:24:48.355502+0000","last_became_active":"2024-09-17T09:24:47.366521+0000","last_became_peered":"2024-09-17T09:24:47.366521+0000","last_unstale":"2024-09-17T09:24:48.355502+0000","last_undegraded":"2024-09-17T09:24:48.355502+0000","last_fullsized":"2024-09-17T09:24:48.355502+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:20:44.637750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"30'19","reported_seq":52,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.649418+0000","last_change":"2024-09-17T09:24:43.338874+0000","last_active":"2024-09-17T09:24:48.649418+0000","last_peered":"2024-09-17T09:24:48.649418+0000","last_clean":"2024-09-17T09:24:48.649418+0000","last_became_active":"2024-09-17T09:24:43.338194+0000","last_became_peered":"2024-09-17T09:24:43.338194+0000","last_unstale":"2024-09-17T09:24:48.649418+0000","last_undegraded":"2024-09-17T09:24:48.649418+0000","last_fullsized":"2024-09-17T09:24:48.649418+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:59:31.118647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.358700+0000","last_change":"2024-09-17T09:24:41.287938+0000","last_active":"2024-09-17T09:24:48.358700+0000","last_peered":"2024-09-17T09:24:48.358700+0000","last_clean":"2024-09-17T09:24:48.358700+0000","last_became_active":"2024-09-17T09:24:41.287681+0000","last_became_peered":"2024-09-17T09:24:41.287681+0000","last_unstale":"2024-09-17T09:24:48.358700+0000","last_undegraded":"2024-09-17T09:24:48.358700+0000","last_fullsized":"2024-09-17T09:24:48.358700+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:17.807274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356978+0000","last_change":"2024-09-17T09:24:45.360582+0000","last_active":"2024-09-17T09:24:48.356978+0000","last_peered":"2024-09-17T09:24:48.356978+0000","last_clean":"2024-09-17T09:24:48.356978+0000","last_became_active":"2024-09-17T09:24:45.360203+0000","last_became_peered":"2024-09-17T09:24:45.360203+0000","last_unstale":"2024-09-17T09:24:48.356978+0000","last_undegraded":"2024-09-17T09:24:48.356978+0000","last_fullsized":"2024-09-17T09:24:48.356978+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:07:32.912332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355908+0000","last_change":"2024-09-17T09:24:47.367503+0000","last_active":"2024-09-17T09:24:48.355908+0000","last_peered":"2024-09-17T09:24:48.355908+0000","last_clean":"2024-09-17T09:24:48.355908+0000","last_became_active":"2024-09-17T09:24:47.367199+0000","last_became_peered":"2024-09-17T09:24:47.367199+0000","last_unstale":"2024-09-17T09:24:48.355908+0000","last_undegraded":"2024-09-17T09:24:48.355908+0000","last_fullsized":"2024-09-17T09:24:48.355908+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:42:31.260557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"30'18","reported_seq":48,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.639215+0000","last_change":"2024-09-17T09:24:43.341211+0000","last_active":"2024-09-17T09:24:48.639215+0000","last_peered":"2024-09-17T09:24:48.639215+0000","last_clean":"2024-09-17T09:24:48.639215+0000","last_became_active":"2024-09-17T09:24:43.340842+0000","last_became_peered":"2024-09-17T09:24:43.340842+0000","last_unstale":"2024-09-17T09:24:48.639215+0000","last_undegraded":"2024-09-17T09:24:48.639215+0000","last_fullsized":"2024-09-17T09:24:48.639215+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:13.076387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355128+0000","last_change":"2024-09-17T09:24:41.286953+0000","last_active":"2024-09-17T09:24:48.355128+0000","last_peered":"2024-09-17T09:24:48.355128+0000","last_clean":"2024-09-17T09:24:48.355128+0000","last_became_active":"2024-09-17T09:24:41.286685+0000","last_became_peered":"2024-09-17T09:24:41.286685+0000","last_unstale":"2024-09-17T09:24:48.355128+0000","last_undegraded":"2024-09-17T09:24:48.355128+0000","last_fullsized":"2024-09-17T09:24:48.355128+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:05.215294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357026+0000","last_change":"2024-09-17T09:24:45.358665+0000","last_active":"2024-09-17T09:24:48.357026+0000","last_peered":"2024-09-17T09:24:48.357026+0000","last_clean":"2024-09-17T09:24:48.357026+0000","last_became_active":"2024-09-17T09:24:45.358271+0000","last_became_peered":"2024-09-17T09:24:45.358271+0000","last_unstale":"2024-09-17T09:24:48.357026+0000","last_undegraded":"2024-09-17T09:24:48.357026+0000","last_fullsized":"2024-09-17T09:24:48.357026+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:51.612758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356435+0000","last_change":"2024-09-17T09:24:47.368684+0000","last_active":"2024-09-17T09:24:48.356435+0000","last_peered":"2024-09-17T09:24:48.356435+0000","last_clean":"2024-09-17T09:24:48.356435+0000","last_became_active":"2024-09-17T09:24:47.368414+0000","last_became_peered":"2024-09-17T09:24:47.368414+0000","last_unstale":"2024-09-17T09:24:48.356435+0000","last_undegraded":"2024-09-17T09:24:48.356435+0000","last_fullsized":"2024-09-17T09:24:48.356435+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:09:23.075065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"30'14","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.667850+0000","last_change":"2024-09-17T09:24:43.341247+0000","last_active":"2024-09-17T09:24:48.667850+0000","last_peered":"2024-09-17T09:24:48.667850+0000","last_clean":"2024-09-17T09:24:48.667850+0000","last_became_active":"2024-09-17T09:24:43.340730+0000","last_became_peered":"2024-09-17T09:24:43.340730+0000","last_unstale":"2024-09-17T09:24:48.667850+0000","last_undegraded":"2024-09-17T09:24:48.667850+0000","last_fullsized":"2024-09-17T09:24:48.667850+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:21:07.874163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.358637+0000","last_change":"2024-09-17T09:24:41.289824+0000","last_active":"2024-09-17T09:24:48.358637+0000","last_peered":"2024-09-17T09:24:48.358637+0000","last_clean":"2024-09-17T09:24:48.358637+0000","last_became_active":"2024-09-17T09:24:41.289494+0000","last_became_peered":"2024-09-17T09:24:41.289494+0000","last_unstale":"2024-09-17T09:24:48.358637+0000","last_undegraded":"2024-09-17T09:24:48.358637+0000","last_fullsized":"2024-09-17T09:24:48.358637+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:01.734882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.358034+0000","last_change":"2024-09-17T09:24:45.356555+0000","last_active":"2024-09-17T09:24:48.358034+0000","last_peered":"2024-09-17T09:24:48.358034+0000","last_clean":"2024-09-17T09:24:48.358034+0000","last_became_active":"2024-09-17T09:24:45.356251+0000","last_became_peered":"2024-09-17T09:24:45.356251+0000","last_unstale":"2024-09-17T09:24:48.358034+0000","last_undegraded":"2024-09-17T09:24:48.358034+0000","last_fullsized":"2024-09-17T09:24:48.358034+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:30.453710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519422+0000","last_change":"2024-09-17T09:24:47.366569+0000","last_active":"2024-09-17T09:24:49.519422+0000","last_peered":"2024-09-17T09:24:49.519422+0000","last_clean":"2024-09-17T09:24:49.519422+0000","last_became_active":"2024-09-17T09:24:47.366181+0000","last_became_peered":"2024-09-17T09:24:47.366181+0000","last_unstale":"2024-09-17T09:24:49.519422+0000","last_undegraded":"2024-09-17T09:24:49.519422+0000","last_fullsized":"2024-09-17T09:24:49.519422+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:03.745415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"30'10","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.655577+0000","last_change":"2024-09-17T09:24:43.339789+0000","last_active":"2024-09-17T09:24:48.655577+0000","last_peered":"2024-09-17T09:24:48.655577+0000","last_clean":"2024-09-17T09:24:48.655577+0000","last_became_active":"2024-09-17T09:24:43.339424+0000","last_became_peered":"2024-09-17T09:24:43.339424+0000","last_unstale":"2024-09-17T09:24:48.655577+0000","last_undegraded":"2024-09-17T09:24:48.655577+0000","last_fullsized":"2024-09-17T09:24:48.655577+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:22.844402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.359414+0000","last_change":"2024-09-17T09:24:41.289128+0000","last_active":"2024-09-17T09:24:48.359414+0000","last_peered":"2024-09-17T09:24:48.359414+0000","last_clean":"2024-09-17T09:24:48.359414+0000","last_became_active":"2024-09-17T09:24:41.288730+0000","last_became_peered":"2024-09-17T09:24:41.288730+0000","last_unstale":"2024-09-17T09:24:48.359414+0000","last_undegraded":"2024-09-17T09:24:48.359414+0000","last_fullsized":"2024-09-17T09:24:48.359414+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:12:44.146430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"31'208","reported_seq":302,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T09:25:08.398893+0000","last_change":"2024-09-17T09:24:28.174643+0000","last_active":"2024-09-17T09:25:08.398893+0000","last_peered":"2024-09-17T09:25:08.398893+0000","last_clean":"2024-09-17T09:25:08.398893+0000","last_became_active":"2024-09-17T09:24:28.173785+0000","last_became_peered":"2024-09-17T09:24:28.173785+0000","last_unstale":"2024-09-17T09:25:08.398893+0000","last_undegraded":"2024-09-17T09:25:08.398893+0000","last_fullsized":"2024-09-17T09:25:08.398893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:27.161582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:27.161582+0000","last_clean_scrub_stamp":"2024-09-17T09:24:27.161582+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:47:41.696981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.359801+0000","last_change":"2024-09-17T09:24:45.357790+0000","last_active":"2024-09-17T09:24:48.359801+0000","last_peered":"2024-09-17T09:24:48.359801+0000","last_clean":"2024-09-17T09:24:48.359801+0000","last_became_active":"2024-09-17T09:24:45.356141+0000","last_became_peered":"2024-09-17T09:24:45.356141+0000","last_unstale":"2024-09-17T09:24:48.359801+0000","last_undegraded":"2024-09-17T09:24:48.359801+0000","last_fullsized":"2024-09-17T09:24:48.359801+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:22:48.481503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356818+0000","last_change":"2024-09-17T09:24:47.362409+0000","last_active":"2024-09-17T09:24:48.356818+0000","last_peered":"2024-09-17T09:24:48.356818+0000","last_clean":"2024-09-17T09:24:48.356818+0000","last_became_active":"2024-09-17T09:24:47.362158+0000","last_became_peered":"2024-09-17T09:24:47.362158+0000","last_unstale":"2024-09-17T09:24:48.356818+0000","last_undegraded":"2024-09-17T09:24:48.356818+0000","last_fullsized":"2024-09-17T09:24:48.356818+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:10.610529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'17","reported_seq":44,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.661486+0000","last_change":"2024-09-17T09:24:43.342925+0000","last_active":"2024-09-17T09:24:48.661486+0000","last_peered":"2024-09-17T09:24:48.661486+0000","last_clean":"2024-09-17T09:24:48.661486+0000","last_became_active":"2024-09-17T09:24:43.342593+0000","last_became_peered":"2024-09-17T09:24:43.342593+0000","last_unstale":"2024-09-17T09:24:48.661486+0000","last_undegraded":"2024-09-17T09:24:48.661486+0000","last_fullsized":"2024-09-17T09:24:48.661486+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:55:10.442236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356729+0000","last_change":"2024-09-17T09:24:41.292578+0000","last_active":"2024-09-17T09:24:48.356729+0000","last_peered":"2024-09-17T09:24:48.356729+0000","last_clean":"2024-09-17T09:24:48.356729+0000","last_became_active":"2024-09-17T09:24:41.291267+0000","last_became_peered":"2024-09-17T09:24:41.291267+0000","last_unstale":"2024-09-17T09:24:48.356729+0000","last_undegraded":"2024-09-17T09:24:48.356729+0000","last_fullsized":"2024-09-17T09:24:48.356729+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:40.184225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.359030+0000","last_change":"2024-09-17T09:24:45.360361+0000","last_active":"2024-09-17T09:24:48.359030+0000","last_peered":"2024-09-17T09:24:48.359030+0000","last_clean":"2024-09-17T09:24:48.359030+0000","last_became_active":"2024-09-17T09:24:45.360043+0000","last_became_peered":"2024-09-17T09:24:45.360043+0000","last_unstale":"2024-09-17T09:24:48.359030+0000","last_undegraded":"2024-09-17T09:24:48.359030+0000","last_fullsized":"2024-09-17T09:24:48.359030+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:18.428377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357109+0000","last_change":"2024-09-17T09:24:47.363468+0000","last_active":"2024-09-17T09:24:48.357109+0000","last_peered":"2024-09-17T09:24:48.357109+0000","last_clean":"2024-09-17T09:24:48.357109+0000","last_became_active":"2024-09-17T09:24:47.363166+0000","last_became_peered":"2024-09-17T09:24:47.363166+0000","last_unstale":"2024-09-17T09:24:48.357109+0000","last_undegraded":"2024-09-17T09:24:48.357109+0000","last_fullsized":"2024-09-17T09:24:48.357109+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:23.724566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"30'10","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.623112+0000","last_change":"2024-09-17T09:24:43.344163+0000","last_active":"2024-09-17T09:24:48.623112+0000","last_peered":"2024-09-17T09:24:48.623112+0000","last_clean":"2024-09-17T09:24:48.623112+0000","last_became_active":"2024-09-17T09:24:43.343881+0000","last_became_peered":"2024-09-17T09:24:43.343881+0000","last_unstale":"2024-09-17T09:24:48.623112+0000","last_undegraded":"2024-09-17T09:24:48.623112+0000","last_fullsized":"2024-09-17T09:24:48.623112+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:50.194582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357574+0000","last_change":"2024-09-17T09:24:41.292696+0000","last_active":"2024-09-17T09:24:48.357574+0000","last_peered":"2024-09-17T09:24:48.357574+0000","last_clean":"2024-09-17T09:24:48.357574+0000","last_became_active":"2024-09-17T09:24:41.291729+0000","last_became_peered":"2024-09-17T09:24:41.291729+0000","last_unstale":"2024-09-17T09:24:48.357574+0000","last_undegraded":"2024-09-17T09:24:48.357574+0000","last_fullsized":"2024-09-17T09:24:48.357574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:07:05.667207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356001+0000","last_change":"2024-09-17T09:24:45.356486+0000","last_active":"2024-09-17T09:24:48.356001+0000","last_peered":"2024-09-17T09:24:48.356001+0000","last_clean":"2024-09-17T09:24:48.356001+0000","last_became_active":"2024-09-17T09:24:45.356111+0000","last_became_peered":"2024-09-17T09:24:45.356111+0000","last_unstale":"2024-09-17T09:24:48.356001+0000","last_undegraded":"2024-09-17T09:24:48.356001+0000","last_fullsized":"2024-09-17T09:24:48.356001+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:22:49.958195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355271+0000","last_change":"2024-09-17T09:24:47.364964+0000","last_active":"2024-09-17T09:24:48.355271+0000","last_peered":"2024-09-17T09:24:48.355271+0000","last_clean":"2024-09-17T09:24:48.355271+0000","last_became_active":"2024-09-17T09:24:47.364664+0000","last_became_peered":"2024-09-17T09:24:47.364664+0000","last_unstale":"2024-09-17T09:24:48.355271+0000","last_undegraded":"2024-09-17T09:24:48.355271+0000","last_fullsized":"2024-09-17T09:24:48.355271+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:16.682260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"30'15","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.684541+0000","last_change":"2024-09-17T09:24:43.343179+0000","last_active":"2024-09-17T09:24:48.684541+0000","last_peered":"2024-09-17T09:24:48.684541+0000","last_clean":"2024-09-17T09:24:48.684541+0000","last_became_active":"2024-09-17T09:24:43.342568+0000","last_became_peered":"2024-09-17T09:24:43.342568+0000","last_unstale":"2024-09-17T09:24:48.684541+0000","last_undegraded":"2024-09-17T09:24:48.684541+0000","last_fullsized":"2024-09-17T09:24:48.684541+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:16:52.632753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356616+0000","last_change":"2024-09-17T09:24:41.289939+0000","last_active":"2024-09-17T09:24:48.356616+0000","last_peered":"2024-09-17T09:24:48.356616+0000","last_clean":"2024-09-17T09:24:48.356616+0000","last_became_active":"2024-09-17T09:24:41.289656+0000","last_became_peered":"2024-09-17T09:24:41.289656+0000","last_unstale":"2024-09-17T09:24:48.356616+0000","last_undegraded":"2024-09-17T09:24:48.356616+0000","last_fullsized":"2024-09-17T09:24:48.356616+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:25.317667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356914+0000","last_change":"2024-09-17T09:24:45.356581+0000","last_active":"2024-09-17T09:24:48.356914+0000","last_peered":"2024-09-17T09:24:48.356914+0000","last_clean":"2024-09-17T09:24:48.356914+0000","last_became_active":"2024-09-17T09:24:45.356325+0000","last_became_peered":"2024-09-17T09:24:45.356325+0000","last_unstale":"2024-09-17T09:24:48.356914+0000","last_undegraded":"2024-09-17T09:24:48.356914+0000","last_fullsized":"2024-09-17T09:24:48.356914+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:38.008051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519155+0000","last_change":"2024-09-17T09:24:47.365586+0000","last_active":"2024-09-17T09:24:49.519155+0000","last_peered":"2024-09-17T09:24:49.519155+0000","last_clean":"2024-09-17T09:24:49.519155+0000","last_became_active":"2024-09-17T09:24:47.365054+0000","last_became_peered":"2024-09-17T09:24:47.365054+0000","last_unstale":"2024-09-17T09:24:49.519155+0000","last_undegraded":"2024-09-17T09:24:49.519155+0000","last_fullsized":"2024-09-17T09:24:49.519155+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:53:31.128315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"30'11","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.663777+0000","last_change":"2024-09-17T09:24:43.345924+0000","last_active":"2024-09-17T09:24:48.663777+0000","last_peered":"2024-09-17T09:24:48.663777+0000","last_clean":"2024-09-17T09:24:48.663777+0000","last_became_active":"2024-09-17T09:24:43.345681+0000","last_became_peered":"2024-09-17T09:24:43.345681+0000","last_unstale":"2024-09-17T09:24:48.663777+0000","last_undegraded":"2024-09-17T09:24:48.663777+0000","last_fullsized":"2024-09-17T09:24:48.663777+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:58:42.327494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"23'2","reported_seq":25,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.467763+0000","last_change":"2024-09-17T09:24:41.292118+0000","last_active":"2024-09-17T09:24:48.467763+0000","last_peered":"2024-09-17T09:24:48.467763+0000","last_clean":"2024-09-17T09:24:48.467763+0000","last_became_active":"2024-09-17T09:24:41.291705+0000","last_became_peered":"2024-09-17T09:24:41.291705+0000","last_unstale":"2024-09-17T09:24:48.467763+0000","last_undegraded":"2024-09-17T09:24:48.467763+0000","last_fullsized":"2024-09-17T09:24:48.467763+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:07:47.644642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"30'8","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T09:25:09.573363+0000","last_change":"2024-09-17T09:24:45.357226+0000","last_active":"2024-09-17T09:25:09.573363+0000","last_peered":"2024-09-17T09:25:09.573363+0000","last_clean":"2024-09-17T09:25:09.573363+0000","last_became_active":"2024-09-17T09:24:45.356987+0000","last_became_peered":"2024-09-17T09:24:45.356987+0000","last_unstale":"2024-09-17T09:25:09.573363+0000","last_undegraded":"2024-09-17T09:25:09.573363+0000","last_fullsized":"2024-09-17T09:25:09.573363+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:25:00.616606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355856+0000","last_change":"2024-09-17T09:24:47.362510+0000","last_active":"2024-09-17T09:24:48.355856+0000","last_peered":"2024-09-17T09:24:48.355856+0000","last_clean":"2024-09-17T09:24:48.355856+0000","last_became_active":"2024-09-17T09:24:47.362156+0000","last_became_peered":"2024-09-17T09:24:47.362156+0000","last_unstale":"2024-09-17T09:24:48.355856+0000","last_undegraded":"2024-09-17T09:24:48.355856+0000","last_fullsized":"2024-09-17T09:24:48.355856+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:55:57.173208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"30'11","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.670358+0000","last_change":"2024-09-17T09:24:43.341286+0000","last_active":"2024-09-17T09:24:48.670358+0000","last_peered":"2024-09-17T09:24:48.670358+0000","last_clean":"2024-09-17T09:24:48.670358+0000","last_became_active":"2024-09-17T09:24:43.340995+0000","last_became_peered":"2024-09-17T09:24:43.340995+0000","last_unstale":"2024-09-17T09:24:48.670358+0000","last_undegraded":"2024-09-17T09:24:48.670358+0000","last_fullsized":"2024-09-17T09:24:48.670358+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:13:44.266571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356140+0000","last_change":"2024-09-17T09:24:41.292908+0000","last_active":"2024-09-17T09:24:48.356140+0000","last_peered":"2024-09-17T09:24:48.356140+0000","last_clean":"2024-09-17T09:24:48.356140+0000","last_became_active":"2024-09-17T09:24:41.292128+0000","last_became_peered":"2024-09-17T09:24:41.292128+0000","last_unstale":"2024-09-17T09:24:48.356140+0000","last_undegraded":"2024-09-17T09:24:48.356140+0000","last_fullsized":"2024-09-17T09:24:48.356140+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:57.939876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357197+0000","last_change":"2024-09-17T09:24:45.358098+0000","last_active":"2024-09-17T09:24:48.357197+0000","last_peered":"2024-09-17T09:24:48.357197+0000","last_clean":"2024-09-17T09:24:48.357197+0000","last_became_active":"2024-09-17T09:24:45.357758+0000","last_became_peered":"2024-09-17T09:24:45.357758+0000","last_unstale":"2024-09-17T09:24:48.357197+0000","last_undegraded":"2024-09-17T09:24:48.357197+0000","last_fullsized":"2024-09-17T09:24:48.357197+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:16.380423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357147+0000","last_change":"2024-09-17T09:24:47.369920+0000","last_active":"2024-09-17T09:24:48.357147+0000","last_peered":"2024-09-17T09:24:48.357147+0000","last_clean":"2024-09-17T09:24:48.357147+0000","last_became_active":"2024-09-17T09:24:47.369616+0000","last_became_peered":"2024-09-17T09:24:47.369616+0000","last_unstale":"2024-09-17T09:24:48.357147+0000","last_undegraded":"2024-09-17T09:24:48.357147+0000","last_fullsized":"2024-09-17T09:24:48.357147+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:02.443979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"30'4","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.508265+0000","last_change":"2024-09-17T09:24:43.344364+0000","last_active":"2024-09-17T09:24:48.508265+0000","last_peered":"2024-09-17T09:24:48.508265+0000","last_clean":"2024-09-17T09:24:48.508265+0000","last_became_active":"2024-09-17T09:24:43.343962+0000","last_became_peered":"2024-09-17T09:24:43.343962+0000","last_unstale":"2024-09-17T09:24:48.508265+0000","last_undegraded":"2024-09-17T09:24:48.508265+0000","last_fullsized":"2024-09-17T09:24:48.508265+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:45:27.731863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.358780+0000","last_change":"2024-09-17T09:24:41.290247+0000","last_active":"2024-09-17T09:24:48.358780+0000","last_peered":"2024-09-17T09:24:48.358780+0000","last_clean":"2024-09-17T09:24:48.358780+0000","last_became_active":"2024-09-17T09:24:41.289562+0000","last_became_peered":"2024-09-17T09:24:41.289562+0000","last_unstale":"2024-09-17T09:24:48.358780+0000","last_undegraded":"2024-09-17T09:24:48.358780+0000","last_fullsized":"2024-09-17T09:24:48.358780+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:14.012718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357405+0000","last_change":"2024-09-17T09:24:45.357891+0000","last_active":"2024-09-17T09:24:48.357405+0000","last_peered":"2024-09-17T09:24:48.357405+0000","last_clean":"2024-09-17T09:24:48.357405+0000","last_became_active":"2024-09-17T09:24:45.357276+0000","last_became_peered":"2024-09-17T09:24:45.357276+0000","last_unstale":"2024-09-17T09:24:48.357405+0000","last_undegraded":"2024-09-17T09:24:48.357405+0000","last_fullsized":"2024-09-17T09:24:48.357405+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:42:38.055142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519370+0000","last_change":"2024-09-17T09:24:47.366422+0000","last_active":"2024-09-17T09:24:49.519370+0000","last_peered":"2024-09-17T09:24:49.519370+0000","last_clean":"2024-09-17T09:24:49.519370+0000","last_became_active":"2024-09-17T09:24:47.366097+0000","last_became_peered":"2024-09-17T09:24:47.366097+0000","last_unstale":"2024-09-17T09:24:49.519370+0000","last_undegraded":"2024-09-17T09:24:49.519370+0000","last_fullsized":"2024-09-17T09:24:49.519370+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:18:09.557860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"30'11","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.642997+0000","last_change":"2024-09-17T09:24:43.343012+0000","last_active":"2024-09-17T09:24:48.642997+0000","last_peered":"2024-09-17T09:24:48.642997+0000","last_clean":"2024-09-17T09:24:48.642997+0000","last_became_active":"2024-09-17T09:24:43.342644+0000","last_became_peered":"2024-09-17T09:24:43.342644+0000","last_unstale":"2024-09-17T09:24:48.642997+0000","last_undegraded":"2024-09-17T09:24:48.642997+0000","last_fullsized":"2024-09-17T09:24:48.642997+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:04.890408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356357+0000","last_change":"2024-09-17T09:24:41.291746+0000","last_active":"2024-09-17T09:24:48.356357+0000","last_peered":"2024-09-17T09:24:48.356357+0000","last_clean":"2024-09-17T09:24:48.356357+0000","last_became_active":"2024-09-17T09:24:41.290651+0000","last_became_peered":"2024-09-17T09:24:41.290651+0000","last_unstale":"2024-09-17T09:24:48.356357+0000","last_undegraded":"2024-09-17T09:24:48.356357+0000","last_fullsized":"2024-09-17T09:24:48.356357+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:30.406366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"30'8","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T09:25:09.573306+0000","last_change":"2024-09-17T09:24:45.357783+0000","last_active":"2024-09-17T09:25:09.573306+0000","last_peered":"2024-09-17T09:25:09.573306+0000","last_clean":"2024-09-17T09:25:09.573306+0000","last_became_active":"2024-09-17T09:24:45.356704+0000","last_became_peered":"2024-09-17T09:24:45.356704+0000","last_unstale":"2024-09-17T09:25:09.573306+0000","last_undegraded":"2024-09-17T09:25:09.573306+0000","last_fullsized":"2024-09-17T09:25:09.573306+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:04:35.792706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519343+0000","last_change":"2024-09-17T09:24:47.365791+0000","last_active":"2024-09-17T09:24:49.519343+0000","last_peered":"2024-09-17T09:24:49.519343+0000","last_clean":"2024-09-17T09:24:49.519343+0000","last_became_active":"2024-09-17T09:24:47.364730+0000","last_became_peered":"2024-09-17T09:24:47.364730+0000","last_unstale":"2024-09-17T09:24:49.519343+0000","last_undegraded":"2024-09-17T09:24:49.519343+0000","last_fullsized":"2024-09-17T09:24:49.519343+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:50:10.156618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.578853+0000","last_change":"2024-09-17T09:24:43.340422+0000","last_active":"2024-09-17T09:24:48.578853+0000","last_peered":"2024-09-17T09:24:48.578853+0000","last_clean":"2024-09-17T09:24:48.578853+0000","last_became_active":"2024-09-17T09:24:43.340091+0000","last_became_peered":"2024-09-17T09:24:43.340091+0000","last_unstale":"2024-09-17T09:24:48.578853+0000","last_undegraded":"2024-09-17T09:24:48.578853+0000","last_fullsized":"2024-09-17T09:24:48.578853+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:43.419138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356535+0000","last_change":"2024-09-17T09:24:41.291627+0000","last_active":"2024-09-17T09:24:48.356535+0000","last_peered":"2024-09-17T09:24:48.356535+0000","last_clean":"2024-09-17T09:24:48.356535+0000","last_became_active":"2024-09-17T09:24:41.291125+0000","last_became_peered":"2024-09-17T09:24:41.291125+0000","last_unstale":"2024-09-17T09:24:48.356535+0000","last_undegraded":"2024-09-17T09:24:48.356535+0000","last_fullsized":"2024-09-17T09:24:48.356535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:45:02.471463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"30'8","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T09:25:09.573295+0000","last_change":"2024-09-17T09:24:45.356466+0000","last_active":"2024-09-17T09:25:09.573295+0000","last_peered":"2024-09-17T09:25:09.573295+0000","last_clean":"2024-09-17T09:25:09.573295+0000","last_became_active":"2024-09-17T09:24:45.356202+0000","last_became_peered":"2024-09-17T09:24:45.356202+0000","last_unstale":"2024-09-17T09:25:09.573295+0000","last_undegraded":"2024-09-17T09:25:09.573295+0000","last_fullsized":"2024-09-17T09:25:09.573295+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:27:32.821368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355928+0000","last_change":"2024-09-17T09:24:47.366655+0000","last_active":"2024-09-17T09:24:48.355928+0000","last_peered":"2024-09-17T09:24:48.355928+0000","last_clean":"2024-09-17T09:24:48.355928+0000","last_became_active":"2024-09-17T09:24:47.366296+0000","last_became_peered":"2024-09-17T09:24:47.366296+0000","last_unstale":"2024-09-17T09:24:48.355928+0000","last_undegraded":"2024-09-17T09:24:48.355928+0000","last_fullsized":"2024-09-17T09:24:48.355928+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:26:10.401509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.653495+0000","last_change":"2024-09-17T09:24:43.341039+0000","last_active":"2024-09-17T09:24:48.653495+0000","last_peered":"2024-09-17T09:24:48.653495+0000","last_clean":"2024-09-17T09:24:48.653495+0000","last_became_active":"2024-09-17T09:24:43.340632+0000","last_became_peered":"2024-09-17T09:24:43.340632+0000","last_unstale":"2024-09-17T09:24:48.653495+0000","last_undegraded":"2024-09-17T09:24:48.653495+0000","last_fullsized":"2024-09-17T09:24:48.653495+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:30.610201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.14","version":"23'1","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.468117+0000","last_change":"2024-09-17T09:24:41.289714+0000","last_active":"2024-09-17T09:24:48.468117+0000","last_peered":"2024-09-17T09:24:48.468117+0000","last_clean":"2024-09-17T09:24:48.468117+0000","last_became_active":"2024-09-17T09:24:41.289175+0000","last_became_peered":"2024-09-17T09:24:41.289175+0000","last_unstale":"2024-09-17T09:24:48.468117+0000","last_undegraded":"2024-09-17T09:24:48.468117+0000","last_fullsized":"2024-09-17T09:24:48.468117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:37.018855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355979+0000","last_change":"2024-09-17T09:24:45.356905+0000","last_active":"2024-09-17T09:24:48.355979+0000","last_peered":"2024-09-17T09:24:48.355979+0000","last_clean":"2024-09-17T09:24:48.355979+0000","last_became_active":"2024-09-17T09:24:45.355852+0000","last_became_peered":"2024-09-17T09:24:45.355852+0000","last_unstale":"2024-09-17T09:24:48.355979+0000","last_undegraded":"2024-09-17T09:24:48.355979+0000","last_fullsized":"2024-09-17T09:24:48.355979+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:05:34.125013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356783+0000","last_change":"2024-09-17T09:24:47.362325+0000","last_active":"2024-09-17T09:24:48.356783+0000","last_peered":"2024-09-17T09:24:48.356783+0000","last_clean":"2024-09-17T09:24:48.356783+0000","last_became_active":"2024-09-17T09:24:47.362038+0000","last_became_peered":"2024-09-17T09:24:47.362038+0000","last_unstale":"2024-09-17T09:24:48.356783+0000","last_undegraded":"2024-09-17T09:24:48.356783+0000","last_fullsized":"2024-09-17T09:24:48.356783+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:56.853743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"30'10","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.651445+0000","last_change":"2024-09-17T09:24:43.343497+0000","last_active":"2024-09-17T09:24:48.651445+0000","last_peered":"2024-09-17T09:24:48.651445+0000","last_clean":"2024-09-17T09:24:48.651445+0000","last_became_active":"2024-09-17T09:24:43.343187+0000","last_became_peered":"2024-09-17T09:24:43.343187+0000","last_unstale":"2024-09-17T09:24:48.651445+0000","last_undegraded":"2024-09-17T09:24:48.651445+0000","last_fullsized":"2024-09-17T09:24:48.651445+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:16.265795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357663+0000","last_change":"2024-09-17T09:24:41.292441+0000","last_active":"2024-09-17T09:24:48.357663+0000","last_peered":"2024-09-17T09:24:48.357663+0000","last_clean":"2024-09-17T09:24:48.357663+0000","last_became_active":"2024-09-17T09:24:41.291371+0000","last_became_peered":"2024-09-17T09:24:41.291371+0000","last_unstale":"2024-09-17T09:24:48.357663+0000","last_undegraded":"2024-09-17T09:24:48.357663+0000","last_fullsized":"2024-09-17T09:24:48.357663+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:23:53.926344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.360298+0000","last_change":"2024-09-17T09:24:45.358331+0000","last_active":"2024-09-17T09:24:48.360298+0000","last_peered":"2024-09-17T09:24:48.360298+0000","last_clean":"2024-09-17T09:24:48.360298+0000","last_became_active":"2024-09-17T09:24:45.356223+0000","last_became_peered":"2024-09-17T09:24:45.356223+0000","last_unstale":"2024-09-17T09:24:48.360298+0000","last_undegraded":"2024-09-17T09:24:48.360298+0000","last_fullsized":"2024-09-17T09:24:48.360298+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:20:17.662397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357354+0000","last_change":"2024-09-17T09:24:47.363473+0000","last_active":"2024-09-17T09:24:48.357354+0000","last_peered":"2024-09-17T09:24:48.357354+0000","last_clean":"2024-09-17T09:24:48.357354+0000","last_became_active":"2024-09-17T09:24:47.363258+0000","last_became_peered":"2024-09-17T09:24:47.363258+0000","last_unstale":"2024-09-17T09:24:48.357354+0000","last_undegraded":"2024-09-17T09:24:48.357354+0000","last_fullsized":"2024-09-17T09:24:48.357354+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:28:11.055372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"30'6","reported_seq":25,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.635027+0000","last_change":"2024-09-17T09:24:43.342800+0000","last_active":"2024-09-17T09:24:48.635027+0000","last_peered":"2024-09-17T09:24:48.635027+0000","last_clean":"2024-09-17T09:24:48.635027+0000","last_became_active":"2024-09-17T09:24:43.341956+0000","last_became_peered":"2024-09-17T09:24:43.341956+0000","last_unstale":"2024-09-17T09:24:48.635027+0000","last_undegraded":"2024-09-17T09:24:48.635027+0000","last_fullsized":"2024-09-17T09:24:48.635027+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:23:43.435775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.360187+0000","last_change":"2024-09-17T09:24:41.290373+0000","last_active":"2024-09-17T09:24:48.360187+0000","last_peered":"2024-09-17T09:24:48.360187+0000","last_clean":"2024-09-17T09:24:48.360187+0000","last_became_active":"2024-09-17T09:24:41.289397+0000","last_became_peered":"2024-09-17T09:24:41.289397+0000","last_unstale":"2024-09-17T09:24:48.360187+0000","last_undegraded":"2024-09-17T09:24:48.360187+0000","last_fullsized":"2024-09-17T09:24:48.360187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:31.436622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355792+0000","last_change":"2024-09-17T09:24:45.360304+0000","last_active":"2024-09-17T09:24:48.355792+0000","last_peered":"2024-09-17T09:24:48.355792+0000","last_clean":"2024-09-17T09:24:48.355792+0000","last_became_active":"2024-09-17T09:24:45.359935+0000","last_became_peered":"2024-09-17T09:24:45.359935+0000","last_unstale":"2024-09-17T09:24:48.355792+0000","last_undegraded":"2024-09-17T09:24:48.355792+0000","last_fullsized":"2024-09-17T09:24:48.355792+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:03:34.460254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519146+0000","last_change":"2024-09-17T09:24:47.365029+0000","last_active":"2024-09-17T09:24:49.519146+0000","last_peered":"2024-09-17T09:24:49.519146+0000","last_clean":"2024-09-17T09:24:49.519146+0000","last_became_active":"2024-09-17T09:24:47.364460+0000","last_became_peered":"2024-09-17T09:24:47.364460+0000","last_unstale":"2024-09-17T09:24:49.519146+0000","last_undegraded":"2024-09-17T09:24:49.519146+0000","last_fullsized":"2024-09-17T09:24:49.519146+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:22.955015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.518888+0000","last_change":"2024-09-17T09:24:43.338583+0000","last_active":"2024-09-17T09:24:48.518888+0000","last_peered":"2024-09-17T09:24:48.518888+0000","last_clean":"2024-09-17T09:24:48.518888+0000","last_became_active":"2024-09-17T09:24:43.337983+0000","last_became_peered":"2024-09-17T09:24:43.337983+0000","last_unstale":"2024-09-17T09:24:48.518888+0000","last_undegraded":"2024-09-17T09:24:48.518888+0000","last_fullsized":"2024-09-17T09:24:48.518888+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:43.090170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.360244+0000","last_change":"2024-09-17T09:24:41.290274+0000","last_active":"2024-09-17T09:24:48.360244+0000","last_peered":"2024-09-17T09:24:48.360244+0000","last_clean":"2024-09-17T09:24:48.360244+0000","last_became_active":"2024-09-17T09:24:41.289094+0000","last_became_peered":"2024-09-17T09:24:41.289094+0000","last_unstale":"2024-09-17T09:24:48.360244+0000","last_undegraded":"2024-09-17T09:24:48.360244+0000","last_fullsized":"2024-09-17T09:24:48.360244+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:07:06.789137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355930+0000","last_change":"2024-09-17T09:24:45.358676+0000","last_active":"2024-09-17T09:24:48.355930+0000","last_peered":"2024-09-17T09:24:48.355930+0000","last_clean":"2024-09-17T09:24:48.355930+0000","last_became_active":"2024-09-17T09:24:45.358193+0000","last_became_peered":"2024-09-17T09:24:45.358193+0000","last_unstale":"2024-09-17T09:24:48.355930+0000","last_undegraded":"2024-09-17T09:24:48.355930+0000","last_fullsized":"2024-09-17T09:24:48.355930+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:48:40.388371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519099+0000","last_change":"2024-09-17T09:24:47.364713+0000","last_active":"2024-09-17T09:24:49.519099+0000","last_peered":"2024-09-17T09:24:49.519099+0000","last_clean":"2024-09-17T09:24:49.519099+0000","last_became_active":"2024-09-17T09:24:47.364345+0000","last_became_peered":"2024-09-17T09:24:47.364345+0000","last_unstale":"2024-09-17T09:24:49.519099+0000","last_undegraded":"2024-09-17T09:24:49.519099+0000","last_fullsized":"2024-09-17T09:24:49.519099+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:34:57.821269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519378+0000","last_change":"2024-09-17T09:24:47.366431+0000","last_active":"2024-09-17T09:24:49.519378+0000","last_peered":"2024-09-17T09:24:49.519378+0000","last_clean":"2024-09-17T09:24:49.519378+0000","last_became_active":"2024-09-17T09:24:47.364476+0000","last_became_peered":"2024-09-17T09:24:47.364476+0000","last_unstale":"2024-09-17T09:24:49.519378+0000","last_undegraded":"2024-09-17T09:24:49.519378+0000","last_fullsized":"2024-09-17T09:24:49.519378+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:02.430539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"30'15","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.676343+0000","last_change":"2024-09-17T09:24:43.341500+0000","last_active":"2024-09-17T09:24:48.676343+0000","last_peered":"2024-09-17T09:24:48.676343+0000","last_clean":"2024-09-17T09:24:48.676343+0000","last_became_active":"2024-09-17T09:24:43.341185+0000","last_became_peered":"2024-09-17T09:24:43.341185+0000","last_unstale":"2024-09-17T09:24:48.676343+0000","last_undegraded":"2024-09-17T09:24:48.676343+0000","last_fullsized":"2024-09-17T09:24:48.676343+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:11.317662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356243+0000","last_change":"2024-09-17T09:24:41.292712+0000","last_active":"2024-09-17T09:24:48.356243+0000","last_peered":"2024-09-17T09:24:48.356243+0000","last_clean":"2024-09-17T09:24:48.356243+0000","last_became_active":"2024-09-17T09:24:41.291236+0000","last_became_peered":"2024-09-17T09:24:41.291236+0000","last_unstale":"2024-09-17T09:24:48.356243+0000","last_undegraded":"2024-09-17T09:24:48.356243+0000","last_fullsized":"2024-09-17T09:24:48.356243+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:42.482034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357330+0000","last_change":"2024-09-17T09:24:45.358111+0000","last_active":"2024-09-17T09:24:48.357330+0000","last_peered":"2024-09-17T09:24:48.357330+0000","last_clean":"2024-09-17T09:24:48.357330+0000","last_became_active":"2024-09-17T09:24:45.357772+0000","last_became_peered":"2024-09-17T09:24:45.357772+0000","last_unstale":"2024-09-17T09:24:48.357330+0000","last_undegraded":"2024-09-17T09:24:48.357330+0000","last_fullsized":"2024-09-17T09:24:48.357330+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:59:23.575217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356333+0000","last_change":"2024-09-17T09:24:47.368096+0000","last_active":"2024-09-17T09:24:48.356333+0000","last_peered":"2024-09-17T09:24:48.356333+0000","last_clean":"2024-09-17T09:24:48.356333+0000","last_became_active":"2024-09-17T09:24:47.367793+0000","last_became_peered":"2024-09-17T09:24:47.367793+0000","last_unstale":"2024-09-17T09:24:48.356333+0000","last_undegraded":"2024-09-17T09:24:48.356333+0000","last_fullsized":"2024-09-17T09:24:48.356333+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:15:08.514704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.488166+0000","last_change":"2024-09-17T09:24:43.342750+0000","last_active":"2024-09-17T09:24:48.488166+0000","last_peered":"2024-09-17T09:24:48.488166+0000","last_clean":"2024-09-17T09:24:48.488166+0000","last_became_active":"2024-09-17T09:24:43.342426+0000","last_became_peered":"2024-09-17T09:24:43.342426+0000","last_unstale":"2024-09-17T09:24:48.488166+0000","last_undegraded":"2024-09-17T09:24:48.488166+0000","last_fullsized":"2024-09-17T09:24:48.488166+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:13:56.840742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355268+0000","last_change":"2024-09-17T09:24:41.289873+0000","last_active":"2024-09-17T09:24:48.355268+0000","last_peered":"2024-09-17T09:24:48.355268+0000","last_clean":"2024-09-17T09:24:48.355268+0000","last_became_active":"2024-09-17T09:24:41.289429+0000","last_became_peered":"2024-09-17T09:24:41.289429+0000","last_unstale":"2024-09-17T09:24:48.355268+0000","last_undegraded":"2024-09-17T09:24:48.355268+0000","last_fullsized":"2024-09-17T09:24:48.355268+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:00.457411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T09:25:09.573357+0000","last_change":"2024-09-17T09:24:45.356796+0000","last_active":"2024-09-17T09:25:09.573357+0000","last_peered":"2024-09-17T09:25:09.573357+0000","last_clean":"2024-09-17T09:25:09.573357+0000","last_became_active":"2024-09-17T09:24:45.356524+0000","last_became_peered":"2024-09-17T09:24:45.356524+0000","last_unstale":"2024-09-17T09:25:09.573357+0000","last_undegraded":"2024-09-17T09:25:09.573357+0000","last_fullsized":"2024-09-17T09:25:09.573357+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:11:50.229474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.358687+0000","last_change":"2024-09-17T09:24:41.288858+0000","last_active":"2024-09-17T09:24:48.358687+0000","last_peered":"2024-09-17T09:24:48.358687+0000","last_clean":"2024-09-17T09:24:48.358687+0000","last_became_active":"2024-09-17T09:24:41.288561+0000","last_became_peered":"2024-09-17T09:24:41.288561+0000","last_unstale":"2024-09-17T09:24:48.358687+0000","last_undegraded":"2024-09-17T09:24:48.358687+0000","last_fullsized":"2024-09-17T09:24:48.358687+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:00.666989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"30'5","reported_seq":26,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.565069+0000","last_change":"2024-09-17T09:24:43.340901+0000","last_active":"2024-09-17T09:24:48.565069+0000","last_peered":"2024-09-17T09:24:48.565069+0000","last_clean":"2024-09-17T09:24:48.565069+0000","last_became_active":"2024-09-17T09:24:43.340593+0000","last_became_peered":"2024-09-17T09:24:43.340593+0000","last_unstale":"2024-09-17T09:24:48.565069+0000","last_undegraded":"2024-09-17T09:24:48.565069+0000","last_fullsized":"2024-09-17T09:24:48.565069+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:23.274367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357500+0000","last_change":"2024-09-17T09:24:45.357747+0000","last_active":"2024-09-17T09:24:48.357500+0000","last_peered":"2024-09-17T09:24:48.357500+0000","last_clean":"2024-09-17T09:24:48.357500+0000","last_became_active":"2024-09-17T09:24:45.356367+0000","last_became_peered":"2024-09-17T09:24:45.356367+0000","last_unstale":"2024-09-17T09:24:48.357500+0000","last_undegraded":"2024-09-17T09:24:48.357500+0000","last_fullsized":"2024-09-17T09:24:48.357500+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:17.465071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355713+0000","last_change":"2024-09-17T09:24:47.368183+0000","last_active":"2024-09-17T09:24:48.355713+0000","last_peered":"2024-09-17T09:24:48.355713+0000","last_clean":"2024-09-17T09:24:48.355713+0000","last_became_active":"2024-09-17T09:24:47.367876+0000","last_became_peered":"2024-09-17T09:24:47.367876+0000","last_unstale":"2024-09-17T09:24:48.355713+0000","last_undegraded":"2024-09-17T09:24:48.355713+0000","last_fullsized":"2024-09-17T09:24:48.355713+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:41.867731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.456917+0000","last_change":"2024-09-17T09:24:41.291466+0000","last_active":"2024-09-17T09:24:48.456917+0000","last_peered":"2024-09-17T09:24:48.456917+0000","last_clean":"2024-09-17T09:24:48.456917+0000","last_became_active":"2024-09-17T09:24:41.290806+0000","last_became_peered":"2024-09-17T09:24:41.290806+0000","last_unstale":"2024-09-17T09:24:48.456917+0000","last_undegraded":"2024-09-17T09:24:48.456917+0000","last_fullsized":"2024-09-17T09:24:48.456917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:08.618737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.585319+0000","last_change":"2024-09-17T09:24:43.340573+0000","last_active":"2024-09-17T09:24:48.585319+0000","last_peered":"2024-09-17T09:24:48.585319+0000","last_clean":"2024-09-17T09:24:48.585319+0000","last_became_active":"2024-09-17T09:24:43.340285+0000","last_became_peered":"2024-09-17T09:24:43.340285+0000","last_unstale":"2024-09-17T09:24:48.585319+0000","last_undegraded":"2024-09-17T09:24:48.585319+0000","last_fullsized":"2024-09-17T09:24:48.585319+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:50.874938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.358122+0000","last_change":"2024-09-17T09:24:45.360363+0000","last_active":"2024-09-17T09:24:48.358122+0000","last_peered":"2024-09-17T09:24:48.358122+0000","last_clean":"2024-09-17T09:24:48.358122+0000","last_became_active":"2024-09-17T09:24:45.360049+0000","last_became_peered":"2024-09-17T09:24:45.360049+0000","last_unstale":"2024-09-17T09:24:48.358122+0000","last_undegraded":"2024-09-17T09:24:48.358122+0000","last_fullsized":"2024-09-17T09:24:48.358122+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:17.291481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519305+0000","last_change":"2024-09-17T09:24:47.365034+0000","last_active":"2024-09-17T09:24:49.519305+0000","last_peered":"2024-09-17T09:24:49.519305+0000","last_clean":"2024-09-17T09:24:49.519305+0000","last_became_active":"2024-09-17T09:24:47.364461+0000","last_became_peered":"2024-09-17T09:24:47.364461+0000","last_unstale":"2024-09-17T09:24:49.519305+0000","last_undegraded":"2024-09-17T09:24:49.519305+0000","last_fullsized":"2024-09-17T09:24:49.519305+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:43.160937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":18,"seq":77309411341,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32132,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708924,"statfs":{"total":95990841344,"available":95957938176,"internally_reserved":0,"allocated":1040384,"data_stored":699308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32136,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708920,"statfs":{"total":95990841344,"available":95957934080,"internally_reserved":0,"allocated":1044480,"data_stored":699977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 09:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.38400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]}]},{"osd":0,"up_from":8,"seq":34359738389,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32132,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708924,"statfs":{"total":95990841344,"available":95957938176,"internally_reserved":0,"allocated":1044480,"data_stored":699977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 09:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.41599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T09:25:22.888 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T09:25:23.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:22 smithi077 ceph-mon[30659]: from='client.14523 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T09:25:23.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:22 smithi003 ceph-mon[25908]: from='client.14523 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T09:25:23.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:22 smithi183 ceph-mon[30372]: from='client.14523 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T09:25:23.188 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:25:24.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:23 smithi077 ceph-mon[30659]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-17T09:25:24.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:23 smithi003 ceph-mon[25908]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-17T09:25:24.008 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:23 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: ::ffff:172.21.15.77 - - [17/Sep/2024:09:25:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T09:25:24.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:23 smithi183 ceph-mon[30372]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-17T09:25:26.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:25 smithi077 ceph-mon[30659]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:26.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:25 smithi003 ceph-mon[25908]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:26.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:25 smithi183 ceph-mon[30372]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:27.573 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:25:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-17T09:25:27.953 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:27 smithi003 ceph-mon[25908]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:28.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:27 smithi077 ceph-mon[30659]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:28.063 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:27 smithi183 ceph-mon[30372]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:28.896 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:28 smithi003 ceph-mon[25908]: from='client.14529 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T09:25:28.897 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-09-17T09:25:26.339916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595838,"num_objects":193,"num_object_clones":0,"num_object_copies":579,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":193,"num_whiteouts":0,"num_read":962,"num_read_kb":809,"num_write":691,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":669,"ondisk_log_size":669,"up":387,"acting":387,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":387,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":281223168,"kb_used":96400,"kb_used_data":3056,"kb_used_omap":4,"kb_used_meta":93307,"kb_avail":281126768,"statfs":{"total":287972524032,"available":287873810432,"internally_reserved":0,"allocated":3129344,"data_stored":2099262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4782,"internal_metadata":95546706},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002225"},"pg_stats":[{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356988+0000","last_change":"2024-09-17T09:24:41.288691+0000","last_active":"2024-09-17T09:24:48.356988+0000","last_peered":"2024-09-17T09:24:48.356988+0000","last_clean":"2024-09-17T09:24:48.356988+0000","last_became_active":"2024-09-17T09:24:41.288367+0000","last_became_peered":"2024-09-17T09:24:41.288367+0000","last_unstale":"2024-09-17T09:24:48.356988+0000","last_undegraded":"2024-09-17T09:24:48.356988+0000","last_fullsized":"2024-09-17T09:24:48.356988+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:25.055167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'10","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.665875+0000","last_change":"2024-09-17T09:24:43.340996+0000","last_active":"2024-09-17T09:24:48.665875+0000","last_peered":"2024-09-17T09:24:48.665875+0000","last_clean":"2024-09-17T09:24:48.665875+0000","last_became_active":"2024-09-17T09:24:43.340648+0000","last_became_peered":"2024-09-17T09:24:43.340648+0000","last_unstale":"2024-09-17T09:24:48.665875+0000","last_undegraded":"2024-09-17T09:24:48.665875+0000","last_fullsized":"2024-09-17T09:24:48.665875+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:29:24.836425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356099+0000","last_change":"2024-09-17T09:24:45.356241+0000","last_active":"2024-09-17T09:24:48.356099+0000","last_peered":"2024-09-17T09:24:48.356099+0000","last_clean":"2024-09-17T09:24:48.356099+0000","last_became_active":"2024-09-17T09:24:45.355977+0000","last_became_peered":"2024-09-17T09:24:45.355977+0000","last_unstale":"2024-09-17T09:24:48.356099+0000","last_undegraded":"2024-09-17T09:24:48.356099+0000","last_fullsized":"2024-09-17T09:24:48.356099+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:03:19.477277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519072+0000","last_change":"2024-09-17T09:24:47.366128+0000","last_active":"2024-09-17T09:24:49.519072+0000","last_peered":"2024-09-17T09:24:49.519072+0000","last_clean":"2024-09-17T09:24:49.519072+0000","last_became_active":"2024-09-17T09:24:47.365112+0000","last_became_peered":"2024-09-17T09:24:47.365112+0000","last_unstale":"2024-09-17T09:24:49.519072+0000","last_undegraded":"2024-09-17T09:24:49.519072+0000","last_fullsized":"2024-09-17T09:24:49.519072+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:29:15.425415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357021+0000","last_change":"2024-09-17T09:24:41.289409+0000","last_active":"2024-09-17T09:24:48.357021+0000","last_peered":"2024-09-17T09:24:48.357021+0000","last_clean":"2024-09-17T09:24:48.357021+0000","last_became_active":"2024-09-17T09:24:41.289152+0000","last_became_peered":"2024-09-17T09:24:41.289152+0000","last_unstale":"2024-09-17T09:24:48.357021+0000","last_undegraded":"2024-09-17T09:24:48.357021+0000","last_fullsized":"2024-09-17T09:24:48.357021+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:41:08.494827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"30'11","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.672303+0000","last_change":"2024-09-17T09:24:43.342559+0000","last_active":"2024-09-17T09:24:48.672303+0000","last_peered":"2024-09-17T09:24:48.672303+0000","last_clean":"2024-09-17T09:24:48.672303+0000","last_became_active":"2024-09-17T09:24:43.342014+0000","last_became_peered":"2024-09-17T09:24:43.342014+0000","last_unstale":"2024-09-17T09:24:48.672303+0000","last_undegraded":"2024-09-17T09:24:48.672303+0000","last_fullsized":"2024-09-17T09:24:48.672303+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:05:26.826249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.358699+0000","last_change":"2024-09-17T09:24:45.361092+0000","last_active":"2024-09-17T09:24:48.358699+0000","last_peered":"2024-09-17T09:24:48.358699+0000","last_clean":"2024-09-17T09:24:48.358699+0000","last_became_active":"2024-09-17T09:24:45.360645+0000","last_became_peered":"2024-09-17T09:24:45.360645+0000","last_unstale":"2024-09-17T09:24:48.358699+0000","last_undegraded":"2024-09-17T09:24:48.358699+0000","last_fullsized":"2024-09-17T09:24:48.358699+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:24.528070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355640+0000","last_change":"2024-09-17T09:24:47.368536+0000","last_active":"2024-09-17T09:24:48.355640+0000","last_peered":"2024-09-17T09:24:48.355640+0000","last_clean":"2024-09-17T09:24:48.355640+0000","last_became_active":"2024-09-17T09:24:47.368224+0000","last_became_peered":"2024-09-17T09:24:47.368224+0000","last_unstale":"2024-09-17T09:24:48.355640+0000","last_undegraded":"2024-09-17T09:24:48.355640+0000","last_fullsized":"2024-09-17T09:24:48.355640+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:26:56.921083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357675+0000","last_change":"2024-09-17T09:24:41.292706+0000","last_active":"2024-09-17T09:24:48.357675+0000","last_peered":"2024-09-17T09:24:48.357675+0000","last_clean":"2024-09-17T09:24:48.357675+0000","last_became_active":"2024-09-17T09:24:41.291355+0000","last_became_peered":"2024-09-17T09:24:41.291355+0000","last_unstale":"2024-09-17T09:24:48.357675+0000","last_undegraded":"2024-09-17T09:24:48.357675+0000","last_fullsized":"2024-09-17T09:24:48.357675+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:19:44.827354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"30'15","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.614974+0000","last_change":"2024-09-17T09:24:43.344325+0000","last_active":"2024-09-17T09:24:48.614974+0000","last_peered":"2024-09-17T09:24:48.614974+0000","last_clean":"2024-09-17T09:24:48.614974+0000","last_became_active":"2024-09-17T09:24:43.344005+0000","last_became_peered":"2024-09-17T09:24:43.344005+0000","last_unstale":"2024-09-17T09:24:48.614974+0000","last_undegraded":"2024-09-17T09:24:48.614974+0000","last_fullsized":"2024-09-17T09:24:48.614974+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:52:17.687949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.360323+0000","last_change":"2024-09-17T09:24:45.358148+0000","last_active":"2024-09-17T09:24:48.360323+0000","last_peered":"2024-09-17T09:24:48.360323+0000","last_clean":"2024-09-17T09:24:48.360323+0000","last_became_active":"2024-09-17T09:24:45.356561+0000","last_became_peered":"2024-09-17T09:24:45.356561+0000","last_unstale":"2024-09-17T09:24:48.360323+0000","last_undegraded":"2024-09-17T09:24:48.360323+0000","last_fullsized":"2024-09-17T09:24:48.360323+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:33.833413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356161+0000","last_change":"2024-09-17T09:24:47.362199+0000","last_active":"2024-09-17T09:24:48.356161+0000","last_peered":"2024-09-17T09:24:48.356161+0000","last_clean":"2024-09-17T09:24:48.356161+0000","last_became_active":"2024-09-17T09:24:47.361927+0000","last_became_peered":"2024-09-17T09:24:47.361927+0000","last_unstale":"2024-09-17T09:24:48.356161+0000","last_undegraded":"2024-09-17T09:24:48.356161+0000","last_fullsized":"2024-09-17T09:24:48.356161+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:18.507166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356935+0000","last_change":"2024-09-17T09:24:41.291521+0000","last_active":"2024-09-17T09:24:48.356935+0000","last_peered":"2024-09-17T09:24:48.356935+0000","last_clean":"2024-09-17T09:24:48.356935+0000","last_became_active":"2024-09-17T09:24:41.291099+0000","last_became_peered":"2024-09-17T09:24:41.291099+0000","last_unstale":"2024-09-17T09:24:48.356935+0000","last_undegraded":"2024-09-17T09:24:48.356935+0000","last_fullsized":"2024-09-17T09:24:48.356935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:58:53.610252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"30'12","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.678612+0000","last_change":"2024-09-17T09:24:43.340649+0000","last_active":"2024-09-17T09:24:48.678612+0000","last_peered":"2024-09-17T09:24:48.678612+0000","last_clean":"2024-09-17T09:24:48.678612+0000","last_became_active":"2024-09-17T09:24:43.340387+0000","last_became_peered":"2024-09-17T09:24:43.340387+0000","last_unstale":"2024-09-17T09:24:48.678612+0000","last_undegraded":"2024-09-17T09:24:48.678612+0000","last_fullsized":"2024-09-17T09:24:48.678612+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:44:44.539535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.359225+0000","last_change":"2024-09-17T09:24:45.361135+0000","last_active":"2024-09-17T09:24:48.359225+0000","last_peered":"2024-09-17T09:24:48.359225+0000","last_clean":"2024-09-17T09:24:48.359225+0000","last_became_active":"2024-09-17T09:24:45.360800+0000","last_became_peered":"2024-09-17T09:24:45.360800+0000","last_unstale":"2024-09-17T09:24:48.359225+0000","last_undegraded":"2024-09-17T09:24:48.359225+0000","last_fullsized":"2024-09-17T09:24:48.359225+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:28:58.469674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519233+0000","last_change":"2024-09-17T09:24:47.365651+0000","last_active":"2024-09-17T09:24:49.519233+0000","last_peered":"2024-09-17T09:24:49.519233+0000","last_clean":"2024-09-17T09:24:49.519233+0000","last_became_active":"2024-09-17T09:24:47.364640+0000","last_became_peered":"2024-09-17T09:24:47.364640+0000","last_unstale":"2024-09-17T09:24:49.519233+0000","last_undegraded":"2024-09-17T09:24:49.519233+0000","last_fullsized":"2024-09-17T09:24:49.519233+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:55:16.538127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"30'19","reported_seq":47,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.682598+0000","last_change":"2024-09-17T09:24:43.343505+0000","last_active":"2024-09-17T09:24:48.682598+0000","last_peered":"2024-09-17T09:24:48.682598+0000","last_clean":"2024-09-17T09:24:48.682598+0000","last_became_active":"2024-09-17T09:24:43.342721+0000","last_became_peered":"2024-09-17T09:24:43.342721+0000","last_unstale":"2024-09-17T09:24:48.682598+0000","last_undegraded":"2024-09-17T09:24:48.682598+0000","last_fullsized":"2024-09-17T09:24:48.682598+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:55.795368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356410+0000","last_change":"2024-09-17T09:24:41.291806+0000","last_active":"2024-09-17T09:24:48.356410+0000","last_peered":"2024-09-17T09:24:48.356410+0000","last_clean":"2024-09-17T09:24:48.356410+0000","last_became_active":"2024-09-17T09:24:41.291541+0000","last_became_peered":"2024-09-17T09:24:41.291541+0000","last_unstale":"2024-09-17T09:24:48.356410+0000","last_undegraded":"2024-09-17T09:24:48.356410+0000","last_fullsized":"2024-09-17T09:24:48.356410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:35:47.673467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"30'8","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T09:25:14.575319+0000","last_change":"2024-09-17T09:24:45.358788+0000","last_active":"2024-09-17T09:25:14.575319+0000","last_peered":"2024-09-17T09:25:14.575319+0000","last_clean":"2024-09-17T09:25:14.575319+0000","last_became_active":"2024-09-17T09:24:45.357990+0000","last_became_peered":"2024-09-17T09:24:45.357990+0000","last_unstale":"2024-09-17T09:25:14.575319+0000","last_undegraded":"2024-09-17T09:25:14.575319+0000","last_fullsized":"2024-09-17T09:25:14.575319+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:50.747097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356001+0000","last_change":"2024-09-17T09:24:47.365848+0000","last_active":"2024-09-17T09:24:48.356001+0000","last_peered":"2024-09-17T09:24:48.356001+0000","last_clean":"2024-09-17T09:24:48.356001+0000","last_became_active":"2024-09-17T09:24:47.365569+0000","last_became_peered":"2024-09-17T09:24:47.365569+0000","last_unstale":"2024-09-17T09:24:48.356001+0000","last_undegraded":"2024-09-17T09:24:48.356001+0000","last_fullsized":"2024-09-17T09:24:48.356001+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:17:10.499880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.645311+0000","last_change":"2024-09-17T09:24:43.339916+0000","last_active":"2024-09-17T09:24:48.645311+0000","last_peered":"2024-09-17T09:24:48.645311+0000","last_clean":"2024-09-17T09:24:48.645311+0000","last_became_active":"2024-09-17T09:24:43.339578+0000","last_became_peered":"2024-09-17T09:24:43.339578+0000","last_unstale":"2024-09-17T09:24:48.645311+0000","last_undegraded":"2024-09-17T09:24:48.645311+0000","last_fullsized":"2024-09-17T09:24:48.645311+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:08:30.859802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356140+0000","last_change":"2024-09-17T09:24:41.292820+0000","last_active":"2024-09-17T09:24:48.356140+0000","last_peered":"2024-09-17T09:24:48.356140+0000","last_clean":"2024-09-17T09:24:48.356140+0000","last_became_active":"2024-09-17T09:24:41.292241+0000","last_became_peered":"2024-09-17T09:24:41.292241+0000","last_unstale":"2024-09-17T09:24:48.356140+0000","last_undegraded":"2024-09-17T09:24:48.356140+0000","last_fullsized":"2024-09-17T09:24:48.356140+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:03.940823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.360291+0000","last_change":"2024-09-17T09:24:45.360696+0000","last_active":"2024-09-17T09:24:48.360291+0000","last_peered":"2024-09-17T09:24:48.360291+0000","last_clean":"2024-09-17T09:24:48.360291+0000","last_became_active":"2024-09-17T09:24:45.360374+0000","last_became_peered":"2024-09-17T09:24:45.360374+0000","last_unstale":"2024-09-17T09:24:48.360291+0000","last_undegraded":"2024-09-17T09:24:48.360291+0000","last_fullsized":"2024-09-17T09:24:48.360291+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:01.952661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.358185+0000","last_change":"2024-09-17T09:24:47.363252+0000","last_active":"2024-09-17T09:24:48.358185+0000","last_peered":"2024-09-17T09:24:48.358185+0000","last_clean":"2024-09-17T09:24:48.358185+0000","last_became_active":"2024-09-17T09:24:47.362918+0000","last_became_peered":"2024-09-17T09:24:47.362918+0000","last_unstale":"2024-09-17T09:24:48.358185+0000","last_undegraded":"2024-09-17T09:24:48.358185+0000","last_fullsized":"2024-09-17T09:24:48.358185+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:37:33.329282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"30'15","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.620920+0000","last_change":"2024-09-17T09:24:43.341292+0000","last_active":"2024-09-17T09:24:48.620920+0000","last_peered":"2024-09-17T09:24:48.620920+0000","last_clean":"2024-09-17T09:24:48.620920+0000","last_became_active":"2024-09-17T09:24:43.341004+0000","last_became_peered":"2024-09-17T09:24:43.341004+0000","last_unstale":"2024-09-17T09:24:48.620920+0000","last_undegraded":"2024-09-17T09:24:48.620920+0000","last_fullsized":"2024-09-17T09:24:48.620920+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:41.425131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355219+0000","last_change":"2024-09-17T09:24:41.289622+0000","last_active":"2024-09-17T09:24:48.355219+0000","last_peered":"2024-09-17T09:24:48.355219+0000","last_clean":"2024-09-17T09:24:48.355219+0000","last_became_active":"2024-09-17T09:24:41.289290+0000","last_became_peered":"2024-09-17T09:24:41.289290+0000","last_unstale":"2024-09-17T09:24:48.355219+0000","last_undegraded":"2024-09-17T09:24:48.355219+0000","last_fullsized":"2024-09-17T09:24:48.355219+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:48:58.205128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357391+0000","last_change":"2024-09-17T09:24:45.356706+0000","last_active":"2024-09-17T09:24:48.357391+0000","last_peered":"2024-09-17T09:24:48.357391+0000","last_clean":"2024-09-17T09:24:48.357391+0000","last_became_active":"2024-09-17T09:24:45.356282+0000","last_became_peered":"2024-09-17T09:24:45.356282+0000","last_unstale":"2024-09-17T09:24:48.357391+0000","last_undegraded":"2024-09-17T09:24:48.357391+0000","last_fullsized":"2024-09-17T09:24:48.357391+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:03.128580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.358527+0000","last_change":"2024-09-17T09:24:47.363118+0000","last_active":"2024-09-17T09:24:48.358527+0000","last_peered":"2024-09-17T09:24:48.358527+0000","last_clean":"2024-09-17T09:24:48.358527+0000","last_became_active":"2024-09-17T09:24:47.362848+0000","last_became_peered":"2024-09-17T09:24:47.362848+0000","last_unstale":"2024-09-17T09:24:48.358527+0000","last_undegraded":"2024-09-17T09:24:48.358527+0000","last_fullsized":"2024-09-17T09:24:48.358527+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:28.334802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"30'12","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.506426+0000","last_change":"2024-09-17T09:24:43.341068+0000","last_active":"2024-09-17T09:24:48.506426+0000","last_peered":"2024-09-17T09:24:48.506426+0000","last_clean":"2024-09-17T09:24:48.506426+0000","last_became_active":"2024-09-17T09:24:43.340760+0000","last_became_peered":"2024-09-17T09:24:43.340760+0000","last_unstale":"2024-09-17T09:24:48.506426+0000","last_undegraded":"2024-09-17T09:24:48.506426+0000","last_fullsized":"2024-09-17T09:24:48.506426+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:16:40.473848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.358725+0000","last_change":"2024-09-17T09:24:41.290572+0000","last_active":"2024-09-17T09:24:48.358725+0000","last_peered":"2024-09-17T09:24:48.358725+0000","last_clean":"2024-09-17T09:24:48.358725+0000","last_became_active":"2024-09-17T09:24:41.290167+0000","last_became_peered":"2024-09-17T09:24:41.290167+0000","last_unstale":"2024-09-17T09:24:48.358725+0000","last_undegraded":"2024-09-17T09:24:48.358725+0000","last_fullsized":"2024-09-17T09:24:48.358725+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:30.727431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"30'8","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T09:25:14.575307+0000","last_change":"2024-09-17T09:24:45.360811+0000","last_active":"2024-09-17T09:25:14.575307+0000","last_peered":"2024-09-17T09:25:14.575307+0000","last_clean":"2024-09-17T09:25:14.575307+0000","last_became_active":"2024-09-17T09:24:45.360518+0000","last_became_peered":"2024-09-17T09:24:45.360518+0000","last_unstale":"2024-09-17T09:25:14.575307+0000","last_undegraded":"2024-09-17T09:25:14.575307+0000","last_fullsized":"2024-09-17T09:25:14.575307+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:00:46.546422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519389+0000","last_change":"2024-09-17T09:24:47.366816+0000","last_active":"2024-09-17T09:24:49.519389+0000","last_peered":"2024-09-17T09:24:49.519389+0000","last_clean":"2024-09-17T09:24:49.519389+0000","last_became_active":"2024-09-17T09:24:47.365986+0000","last_became_peered":"2024-09-17T09:24:47.365986+0000","last_unstale":"2024-09-17T09:24:49.519389+0000","last_undegraded":"2024-09-17T09:24:49.519389+0000","last_fullsized":"2024-09-17T09:24:49.519389+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:39.707829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"30'12","reported_seq":34,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.613115+0000","last_change":"2024-09-17T09:24:43.339922+0000","last_active":"2024-09-17T09:24:48.613115+0000","last_peered":"2024-09-17T09:24:48.613115+0000","last_clean":"2024-09-17T09:24:48.613115+0000","last_became_active":"2024-09-17T09:24:43.339504+0000","last_became_peered":"2024-09-17T09:24:43.339504+0000","last_unstale":"2024-09-17T09:24:48.613115+0000","last_undegraded":"2024-09-17T09:24:48.613115+0000","last_fullsized":"2024-09-17T09:24:48.613115+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:56.919668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.360092+0000","last_change":"2024-09-17T09:24:41.286119+0000","last_active":"2024-09-17T09:24:48.360092+0000","last_peered":"2024-09-17T09:24:48.360092+0000","last_clean":"2024-09-17T09:24:48.360092+0000","last_became_active":"2024-09-17T09:24:41.285741+0000","last_became_peered":"2024-09-17T09:24:41.285741+0000","last_unstale":"2024-09-17T09:24:48.360092+0000","last_undegraded":"2024-09-17T09:24:48.360092+0000","last_fullsized":"2024-09-17T09:24:48.360092+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:03:08.366768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356488+0000","last_change":"2024-09-17T09:24:45.357383+0000","last_active":"2024-09-17T09:24:48.356488+0000","last_peered":"2024-09-17T09:24:48.356488+0000","last_clean":"2024-09-17T09:24:48.356488+0000","last_became_active":"2024-09-17T09:24:45.357100+0000","last_became_peered":"2024-09-17T09:24:45.357100+0000","last_unstale":"2024-09-17T09:24:48.356488+0000","last_undegraded":"2024-09-17T09:24:48.356488+0000","last_fullsized":"2024-09-17T09:24:48.356488+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:39:44.780737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355865+0000","last_change":"2024-09-17T09:24:47.363041+0000","last_active":"2024-09-17T09:24:48.355865+0000","last_peered":"2024-09-17T09:24:48.355865+0000","last_clean":"2024-09-17T09:24:48.355865+0000","last_became_active":"2024-09-17T09:24:47.362691+0000","last_became_peered":"2024-09-17T09:24:47.362691+0000","last_unstale":"2024-09-17T09:24:48.355865+0000","last_undegraded":"2024-09-17T09:24:48.355865+0000","last_fullsized":"2024-09-17T09:24:48.355865+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:40:21.814776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"30'13","reported_seq":43,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.603883+0000","last_change":"2024-09-17T09:24:43.341513+0000","last_active":"2024-09-17T09:24:48.603883+0000","last_peered":"2024-09-17T09:24:48.603883+0000","last_clean":"2024-09-17T09:24:48.603883+0000","last_became_active":"2024-09-17T09:24:43.341100+0000","last_became_peered":"2024-09-17T09:24:43.341100+0000","last_unstale":"2024-09-17T09:24:48.603883+0000","last_undegraded":"2024-09-17T09:24:48.603883+0000","last_fullsized":"2024-09-17T09:24:48.603883+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:48:44.287485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356626+0000","last_change":"2024-09-17T09:24:41.286001+0000","last_active":"2024-09-17T09:24:48.356626+0000","last_peered":"2024-09-17T09:24:48.356626+0000","last_clean":"2024-09-17T09:24:48.356626+0000","last_became_active":"2024-09-17T09:24:41.285710+0000","last_became_peered":"2024-09-17T09:24:41.285710+0000","last_unstale":"2024-09-17T09:24:48.356626+0000","last_undegraded":"2024-09-17T09:24:48.356626+0000","last_fullsized":"2024-09-17T09:24:48.356626+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:41.623205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"30'8","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T09:25:14.575118+0000","last_change":"2024-09-17T09:24:45.359134+0000","last_active":"2024-09-17T09:25:14.575118+0000","last_peered":"2024-09-17T09:25:14.575118+0000","last_clean":"2024-09-17T09:25:14.575118+0000","last_became_active":"2024-09-17T09:24:45.358758+0000","last_became_peered":"2024-09-17T09:24:45.358758+0000","last_unstale":"2024-09-17T09:25:14.575118+0000","last_undegraded":"2024-09-17T09:25:14.575118+0000","last_fullsized":"2024-09-17T09:25:14.575118+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:03.918860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519174+0000","last_change":"2024-09-17T09:24:47.364802+0000","last_active":"2024-09-17T09:24:49.519174+0000","last_peered":"2024-09-17T09:24:49.519174+0000","last_clean":"2024-09-17T09:24:49.519174+0000","last_became_active":"2024-09-17T09:24:47.363953+0000","last_became_peered":"2024-09-17T09:24:47.363953+0000","last_unstale":"2024-09-17T09:24:49.519174+0000","last_undegraded":"2024-09-17T09:24:49.519174+0000","last_fullsized":"2024-09-17T09:24:49.519174+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:49:20.868575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"30'28","reported_seq":72,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T09:25:14.575121+0000","last_change":"2024-09-17T09:24:43.342381+0000","last_active":"2024-09-17T09:25:14.575121+0000","last_peered":"2024-09-17T09:25:14.575121+0000","last_clean":"2024-09-17T09:25:14.575121+0000","last_became_active":"2024-09-17T09:24:43.342031+0000","last_became_peered":"2024-09-17T09:24:43.342031+0000","last_unstale":"2024-09-17T09:25:14.575121+0000","last_undegraded":"2024-09-17T09:25:14.575121+0000","last_fullsized":"2024-09-17T09:25:14.575121+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:28:42.373131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357456+0000","last_change":"2024-09-17T09:24:41.292908+0000","last_active":"2024-09-17T09:24:48.357456+0000","last_peered":"2024-09-17T09:24:48.357456+0000","last_clean":"2024-09-17T09:24:48.357456+0000","last_became_active":"2024-09-17T09:24:41.292251+0000","last_became_peered":"2024-09-17T09:24:41.292251+0000","last_unstale":"2024-09-17T09:24:48.357456+0000","last_undegraded":"2024-09-17T09:24:48.357456+0000","last_fullsized":"2024-09-17T09:24:48.357456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:46.175913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"30'8","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T09:25:14.575361+0000","last_change":"2024-09-17T09:24:45.355662+0000","last_active":"2024-09-17T09:25:14.575361+0000","last_peered":"2024-09-17T09:25:14.575361+0000","last_clean":"2024-09-17T09:25:14.575361+0000","last_became_active":"2024-09-17T09:24:45.355400+0000","last_became_peered":"2024-09-17T09:24:45.355400+0000","last_unstale":"2024-09-17T09:25:14.575361+0000","last_undegraded":"2024-09-17T09:25:14.575361+0000","last_fullsized":"2024-09-17T09:25:14.575361+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:35.789402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519210+0000","last_change":"2024-09-17T09:24:47.366360+0000","last_active":"2024-09-17T09:24:49.519210+0000","last_peered":"2024-09-17T09:24:49.519210+0000","last_clean":"2024-09-17T09:24:49.519210+0000","last_became_active":"2024-09-17T09:24:47.366028+0000","last_became_peered":"2024-09-17T09:24:47.366028+0000","last_unstale":"2024-09-17T09:24:49.519210+0000","last_undegraded":"2024-09-17T09:24:49.519210+0000","last_fullsized":"2024-09-17T09:24:49.519210+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:28.403235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"30'14","reported_seq":38,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.659659+0000","last_change":"2024-09-17T09:24:43.344659+0000","last_active":"2024-09-17T09:24:48.659659+0000","last_peered":"2024-09-17T09:24:48.659659+0000","last_clean":"2024-09-17T09:24:48.659659+0000","last_became_active":"2024-09-17T09:24:43.344367+0000","last_became_peered":"2024-09-17T09:24:43.344367+0000","last_unstale":"2024-09-17T09:24:48.659659+0000","last_undegraded":"2024-09-17T09:24:48.659659+0000","last_fullsized":"2024-09-17T09:24:48.659659+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:35.398373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356055+0000","last_change":"2024-09-17T09:24:41.288895+0000","last_active":"2024-09-17T09:24:48.356055+0000","last_peered":"2024-09-17T09:24:48.356055+0000","last_clean":"2024-09-17T09:24:48.356055+0000","last_became_active":"2024-09-17T09:24:41.288403+0000","last_became_peered":"2024-09-17T09:24:41.288403+0000","last_unstale":"2024-09-17T09:24:48.356055+0000","last_undegraded":"2024-09-17T09:24:48.356055+0000","last_fullsized":"2024-09-17T09:24:48.356055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:20:01.176505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357256+0000","last_change":"2024-09-17T09:24:45.357655+0000","last_active":"2024-09-17T09:24:48.357256+0000","last_peered":"2024-09-17T09:24:48.357256+0000","last_clean":"2024-09-17T09:24:48.357256+0000","last_became_active":"2024-09-17T09:24:45.357388+0000","last_became_peered":"2024-09-17T09:24:45.357388+0000","last_unstale":"2024-09-17T09:24:48.357256+0000","last_undegraded":"2024-09-17T09:24:48.357256+0000","last_fullsized":"2024-09-17T09:24:48.357256+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:25:05.989337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355502+0000","last_change":"2024-09-17T09:24:47.366854+0000","last_active":"2024-09-17T09:24:48.355502+0000","last_peered":"2024-09-17T09:24:48.355502+0000","last_clean":"2024-09-17T09:24:48.355502+0000","last_became_active":"2024-09-17T09:24:47.366521+0000","last_became_peered":"2024-09-17T09:24:47.366521+0000","last_unstale":"2024-09-17T09:24:48.355502+0000","last_undegraded":"2024-09-17T09:24:48.355502+0000","last_fullsized":"2024-09-17T09:24:48.355502+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:20:44.637750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"30'19","reported_seq":52,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.649418+0000","last_change":"2024-09-17T09:24:43.338874+0000","last_active":"2024-09-17T09:24:48.649418+0000","last_peered":"2024-09-17T09:24:48.649418+0000","last_clean":"2024-09-17T09:24:48.649418+0000","last_became_active":"2024-09-17T09:24:43.338194+0000","last_became_peered":"2024-09-17T09:24:43.338194+0000","last_unstale":"2024-09-17T09:24:48.649418+0000","last_undegraded":"2024-09-17T09:24:48.649418+0000","last_fullsized":"2024-09-17T09:24:48.649418+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:59:31.118647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.358700+0000","last_change":"2024-09-17T09:24:41.287938+0000","last_active":"2024-09-17T09:24:48.358700+0000","last_peered":"2024-09-17T09:24:48.358700+0000","last_clean":"2024-09-17T09:24:48.358700+0000","last_became_active":"2024-09-17T09:24:41.287681+0000","last_became_peered":"2024-09-17T09:24:41.287681+0000","last_unstale":"2024-09-17T09:24:48.358700+0000","last_undegraded":"2024-09-17T09:24:48.358700+0000","last_fullsized":"2024-09-17T09:24:48.358700+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:17.807274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356978+0000","last_change":"2024-09-17T09:24:45.360582+0000","last_active":"2024-09-17T09:24:48.356978+0000","last_peered":"2024-09-17T09:24:48.356978+0000","last_clean":"2024-09-17T09:24:48.356978+0000","last_became_active":"2024-09-17T09:24:45.360203+0000","last_became_peered":"2024-09-17T09:24:45.360203+0000","last_unstale":"2024-09-17T09:24:48.356978+0000","last_undegraded":"2024-09-17T09:24:48.356978+0000","last_fullsized":"2024-09-17T09:24:48.356978+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:07:32.912332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355908+0000","last_change":"2024-09-17T09:24:47.367503+0000","last_active":"2024-09-17T09:24:48.355908+0000","last_peered":"2024-09-17T09:24:48.355908+0000","last_clean":"2024-09-17T09:24:48.355908+0000","last_became_active":"2024-09-17T09:24:47.367199+0000","last_became_peered":"2024-09-17T09:24:47.367199+0000","last_unstale":"2024-09-17T09:24:48.355908+0000","last_undegraded":"2024-09-17T09:24:48.355908+0000","last_fullsized":"2024-09-17T09:24:48.355908+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:42:31.260557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"30'18","reported_seq":48,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.639215+0000","last_change":"2024-09-17T09:24:43.341211+0000","last_active":"2024-09-17T09:24:48.639215+0000","last_peered":"2024-09-17T09:24:48.639215+0000","last_clean":"2024-09-17T09:24:48.639215+0000","last_became_active":"2024-09-17T09:24:43.340842+0000","last_became_peered":"2024-09-17T09:24:43.340842+0000","last_unstale":"2024-09-17T09:24:48.639215+0000","last_undegraded":"2024-09-17T09:24:48.639215+0000","last_fullsized":"2024-09-17T09:24:48.639215+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:13.076387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355128+0000","last_change":"2024-09-17T09:24:41.286953+0000","last_active":"2024-09-17T09:24:48.355128+0000","last_peered":"2024-09-17T09:24:48.355128+0000","last_clean":"2024-09-17T09:24:48.355128+0000","last_became_active":"2024-09-17T09:24:41.286685+0000","last_became_peered":"2024-09-17T09:24:41.286685+0000","last_unstale":"2024-09-17T09:24:48.355128+0000","last_undegraded":"2024-09-17T09:24:48.355128+0000","last_fullsized":"2024-09-17T09:24:48.355128+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:05.215294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357026+0000","last_change":"2024-09-17T09:24:45.358665+0000","last_active":"2024-09-17T09:24:48.357026+0000","last_peered":"2024-09-17T09:24:48.357026+0000","last_clean":"2024-09-17T09:24:48.357026+0000","last_became_active":"2024-09-17T09:24:45.358271+0000","last_became_peered":"2024-09-17T09:24:45.358271+0000","last_unstale":"2024-09-17T09:24:48.357026+0000","last_undegraded":"2024-09-17T09:24:48.357026+0000","last_fullsized":"2024-09-17T09:24:48.357026+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:51.612758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356435+0000","last_change":"2024-09-17T09:24:47.368684+0000","last_active":"2024-09-17T09:24:48.356435+0000","last_peered":"2024-09-17T09:24:48.356435+0000","last_clean":"2024-09-17T09:24:48.356435+0000","last_became_active":"2024-09-17T09:24:47.368414+0000","last_became_peered":"2024-09-17T09:24:47.368414+0000","last_unstale":"2024-09-17T09:24:48.356435+0000","last_undegraded":"2024-09-17T09:24:48.356435+0000","last_fullsized":"2024-09-17T09:24:48.356435+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:09:23.075065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"30'14","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.667850+0000","last_change":"2024-09-17T09:24:43.341247+0000","last_active":"2024-09-17T09:24:48.667850+0000","last_peered":"2024-09-17T09:24:48.667850+0000","last_clean":"2024-09-17T09:24:48.667850+0000","last_became_active":"2024-09-17T09:24:43.340730+0000","last_became_peered":"2024-09-17T09:24:43.340730+0000","last_unstale":"2024-09-17T09:24:48.667850+0000","last_undegraded":"2024-09-17T09:24:48.667850+0000","last_fullsized":"2024-09-17T09:24:48.667850+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:21:07.874163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.358637+0000","last_change":"2024-09-17T09:24:41.289824+0000","last_active":"2024-09-17T09:24:48.358637+0000","last_peered":"2024-09-17T09:24:48.358637+0000","last_clean":"2024-09-17T09:24:48.358637+0000","last_became_active":"2024-09-17T09:24:41.289494+0000","last_became_peered":"2024-09-17T09:24:41.289494+0000","last_unstale":"2024-09-17T09:24:48.358637+0000","last_undegraded":"2024-09-17T09:24:48.358637+0000","last_fullsized":"2024-09-17T09:24:48.358637+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:01.734882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.358034+0000","last_change":"2024-09-17T09:24:45.356555+0000","last_active":"2024-09-17T09:24:48.358034+0000","last_peered":"2024-09-17T09:24:48.358034+0000","last_clean":"2024-09-17T09:24:48.358034+0000","last_became_active":"2024-09-17T09:24:45.356251+0000","last_became_peered":"2024-09-17T09:24:45.356251+0000","last_unstale":"2024-09-17T09:24:48.358034+0000","last_undegraded":"2024-09-17T09:24:48.358034+0000","last_fullsized":"2024-09-17T09:24:48.358034+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:30.453710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519422+0000","last_change":"2024-09-17T09:24:47.366569+0000","last_active":"2024-09-17T09:24:49.519422+0000","last_peered":"2024-09-17T09:24:49.519422+0000","last_clean":"2024-09-17T09:24:49.519422+0000","last_became_active":"2024-09-17T09:24:47.366181+0000","last_became_peered":"2024-09-17T09:24:47.366181+0000","last_unstale":"2024-09-17T09:24:49.519422+0000","last_undegraded":"2024-09-17T09:24:49.519422+0000","last_fullsized":"2024-09-17T09:24:49.519422+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:03.745415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"30'10","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.655577+0000","last_change":"2024-09-17T09:24:43.339789+0000","last_active":"2024-09-17T09:24:48.655577+0000","last_peered":"2024-09-17T09:24:48.655577+0000","last_clean":"2024-09-17T09:24:48.655577+0000","last_became_active":"2024-09-17T09:24:43.339424+0000","last_became_peered":"2024-09-17T09:24:43.339424+0000","last_unstale":"2024-09-17T09:24:48.655577+0000","last_undegraded":"2024-09-17T09:24:48.655577+0000","last_fullsized":"2024-09-17T09:24:48.655577+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:22.844402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.359414+0000","last_change":"2024-09-17T09:24:41.289128+0000","last_active":"2024-09-17T09:24:48.359414+0000","last_peered":"2024-09-17T09:24:48.359414+0000","last_clean":"2024-09-17T09:24:48.359414+0000","last_became_active":"2024-09-17T09:24:41.288730+0000","last_became_peered":"2024-09-17T09:24:41.288730+0000","last_unstale":"2024-09-17T09:24:48.359414+0000","last_undegraded":"2024-09-17T09:24:48.359414+0000","last_fullsized":"2024-09-17T09:24:48.359414+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:12:44.146430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"31'208","reported_seq":302,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T09:25:08.398893+0000","last_change":"2024-09-17T09:24:28.174643+0000","last_active":"2024-09-17T09:25:08.398893+0000","last_peered":"2024-09-17T09:25:08.398893+0000","last_clean":"2024-09-17T09:25:08.398893+0000","last_became_active":"2024-09-17T09:24:28.173785+0000","last_became_peered":"2024-09-17T09:24:28.173785+0000","last_unstale":"2024-09-17T09:25:08.398893+0000","last_undegraded":"2024-09-17T09:25:08.398893+0000","last_fullsized":"2024-09-17T09:25:08.398893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:27.161582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:27.161582+0000","last_clean_scrub_stamp":"2024-09-17T09:24:27.161582+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:47:41.696981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.359801+0000","last_change":"2024-09-17T09:24:45.357790+0000","last_active":"2024-09-17T09:24:48.359801+0000","last_peered":"2024-09-17T09:24:48.359801+0000","last_clean":"2024-09-17T09:24:48.359801+0000","last_became_active":"2024-09-17T09:24:45.356141+0000","last_became_peered":"2024-09-17T09:24:45.356141+0000","last_unstale":"2024-09-17T09:24:48.359801+0000","last_undegraded":"2024-09-17T09:24:48.359801+0000","last_fullsized":"2024-09-17T09:24:48.359801+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:22:48.481503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356818+0000","last_change":"2024-09-17T09:24:47.362409+0000","last_active":"2024-09-17T09:24:48.356818+0000","last_peered":"2024-09-17T09:24:48.356818+0000","last_clean":"2024-09-17T09:24:48.356818+0000","last_became_active":"2024-09-17T09:24:47.362158+0000","last_became_peered":"2024-09-17T09:24:47.362158+0000","last_unstale":"2024-09-17T09:24:48.356818+0000","last_undegraded":"2024-09-17T09:24:48.356818+0000","last_fullsized":"2024-09-17T09:24:48.356818+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:10.610529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'17","reported_seq":44,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.661486+0000","last_change":"2024-09-17T09:24:43.342925+0000","last_active":"2024-09-17T09:24:48.661486+0000","last_peered":"2024-09-17T09:24:48.661486+0000","last_clean":"2024-09-17T09:24:48.661486+0000","last_became_active":"2024-09-17T09:24:43.342593+0000","last_became_peered":"2024-09-17T09:24:43.342593+0000","last_unstale":"2024-09-17T09:24:48.661486+0000","last_undegraded":"2024-09-17T09:24:48.661486+0000","last_fullsized":"2024-09-17T09:24:48.661486+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:55:10.442236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356729+0000","last_change":"2024-09-17T09:24:41.292578+0000","last_active":"2024-09-17T09:24:48.356729+0000","last_peered":"2024-09-17T09:24:48.356729+0000","last_clean":"2024-09-17T09:24:48.356729+0000","last_became_active":"2024-09-17T09:24:41.291267+0000","last_became_peered":"2024-09-17T09:24:41.291267+0000","last_unstale":"2024-09-17T09:24:48.356729+0000","last_undegraded":"2024-09-17T09:24:48.356729+0000","last_fullsized":"2024-09-17T09:24:48.356729+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:40.184225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.359030+0000","last_change":"2024-09-17T09:24:45.360361+0000","last_active":"2024-09-17T09:24:48.359030+0000","last_peered":"2024-09-17T09:24:48.359030+0000","last_clean":"2024-09-17T09:24:48.359030+0000","last_became_active":"2024-09-17T09:24:45.360043+0000","last_became_peered":"2024-09-17T09:24:45.360043+0000","last_unstale":"2024-09-17T09:24:48.359030+0000","last_undegraded":"2024-09-17T09:24:48.359030+0000","last_fullsized":"2024-09-17T09:24:48.359030+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:18.428377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357109+0000","last_change":"2024-09-17T09:24:47.363468+0000","last_active":"2024-09-17T09:24:48.357109+0000","last_peered":"2024-09-17T09:24:48.357109+0000","last_clean":"2024-09-17T09:24:48.357109+0000","last_became_active":"2024-09-17T09:24:47.363166+0000","last_became_peered":"2024-09-17T09:24:47.363166+0000","last_unstale":"2024-09-17T09:24:48.357109+0000","last_undegraded":"2024-09-17T09:24:48.357109+0000","last_fullsized":"2024-09-17T09:24:48.357109+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:23.724566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"30'10","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.623112+0000","last_change":"2024-09-17T09:24:43.344163+0000","last_active":"2024-09-17T09:24:48.623112+0000","last_peered":"2024-09-17T09:24:48.623112+0000","last_clean":"2024-09-17T09:24:48.623112+0000","last_became_active":"2024-09-17T09:24:43.343881+0000","last_became_peered":"2024-09-17T09:24:43.343881+0000","last_unstale":"2024-09-17T09:24:48.623112+0000","last_undegraded":"2024-09-17T09:24:48.623112+0000","last_fullsized":"2024-09-17T09:24:48.623112+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:50.194582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357574+0000","last_change":"2024-09-17T09:24:41.292696+0000","last_active":"2024-09-17T09:24:48.357574+0000","last_peered":"2024-09-17T09:24:48.357574+0000","last_clean":"2024-09-17T09:24:48.357574+0000","last_became_active":"2024-09-17T09:24:41.291729+0000","last_became_peered":"2024-09-17T09:24:41.291729+0000","last_unstale":"2024-09-17T09:24:48.357574+0000","last_undegraded":"2024-09-17T09:24:48.357574+0000","last_fullsized":"2024-09-17T09:24:48.357574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:07:05.667207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356001+0000","last_change":"2024-09-17T09:24:45.356486+0000","last_active":"2024-09-17T09:24:48.356001+0000","last_peered":"2024-09-17T09:24:48.356001+0000","last_clean":"2024-09-17T09:24:48.356001+0000","last_became_active":"2024-09-17T09:24:45.356111+0000","last_became_peered":"2024-09-17T09:24:45.356111+0000","last_unstale":"2024-09-17T09:24:48.356001+0000","last_undegraded":"2024-09-17T09:24:48.356001+0000","last_fullsized":"2024-09-17T09:24:48.356001+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:22:49.958195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355271+0000","last_change":"2024-09-17T09:24:47.364964+0000","last_active":"2024-09-17T09:24:48.355271+0000","last_peered":"2024-09-17T09:24:48.355271+0000","last_clean":"2024-09-17T09:24:48.355271+0000","last_became_active":"2024-09-17T09:24:47.364664+0000","last_became_peered":"2024-09-17T09:24:47.364664+0000","last_unstale":"2024-09-17T09:24:48.355271+0000","last_undegraded":"2024-09-17T09:24:48.355271+0000","last_fullsized":"2024-09-17T09:24:48.355271+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:16.682260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"30'15","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.684541+0000","last_change":"2024-09-17T09:24:43.343179+0000","last_active":"2024-09-17T09:24:48.684541+0000","last_peered":"2024-09-17T09:24:48.684541+0000","last_clean":"2024-09-17T09:24:48.684541+0000","last_became_active":"2024-09-17T09:24:43.342568+0000","last_became_peered":"2024-09-17T09:24:43.342568+0000","last_unstale":"2024-09-17T09:24:48.684541+0000","last_undegraded":"2024-09-17T09:24:48.684541+0000","last_fullsized":"2024-09-17T09:24:48.684541+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:16:52.632753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356616+0000","last_change":"2024-09-17T09:24:41.289939+0000","last_active":"2024-09-17T09:24:48.356616+0000","last_peered":"2024-09-17T09:24:48.356616+0000","last_clean":"2024-09-17T09:24:48.356616+0000","last_became_active":"2024-09-17T09:24:41.289656+0000","last_became_peered":"2024-09-17T09:24:41.289656+0000","last_unstale":"2024-09-17T09:24:48.356616+0000","last_undegraded":"2024-09-17T09:24:48.356616+0000","last_fullsized":"2024-09-17T09:24:48.356616+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:25.317667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356914+0000","last_change":"2024-09-17T09:24:45.356581+0000","last_active":"2024-09-17T09:24:48.356914+0000","last_peered":"2024-09-17T09:24:48.356914+0000","last_clean":"2024-09-17T09:24:48.356914+0000","last_became_active":"2024-09-17T09:24:45.356325+0000","last_became_peered":"2024-09-17T09:24:45.356325+0000","last_unstale":"2024-09-17T09:24:48.356914+0000","last_undegraded":"2024-09-17T09:24:48.356914+0000","last_fullsized":"2024-09-17T09:24:48.356914+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:38.008051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519155+0000","last_change":"2024-09-17T09:24:47.365586+0000","last_active":"2024-09-17T09:24:49.519155+0000","last_peered":"2024-09-17T09:24:49.519155+0000","last_clean":"2024-09-17T09:24:49.519155+0000","last_became_active":"2024-09-17T09:24:47.365054+0000","last_became_peered":"2024-09-17T09:24:47.365054+0000","last_unstale":"2024-09-17T09:24:49.519155+0000","last_undegraded":"2024-09-17T09:24:49.519155+0000","last_fullsized":"2024-09-17T09:24:49.519155+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:53:31.128315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"30'11","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.663777+0000","last_change":"2024-09-17T09:24:43.345924+0000","last_active":"2024-09-17T09:24:48.663777+0000","last_peered":"2024-09-17T09:24:48.663777+0000","last_clean":"2024-09-17T09:24:48.663777+0000","last_became_active":"2024-09-17T09:24:43.345681+0000","last_became_peered":"2024-09-17T09:24:43.345681+0000","last_unstale":"2024-09-17T09:24:48.663777+0000","last_undegraded":"2024-09-17T09:24:48.663777+0000","last_fullsized":"2024-09-17T09:24:48.663777+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:58:42.327494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"23'2","reported_seq":25,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.467763+0000","last_change":"2024-09-17T09:24:41.292118+0000","last_active":"2024-09-17T09:24:48.467763+0000","last_peered":"2024-09-17T09:24:48.467763+0000","last_clean":"2024-09-17T09:24:48.467763+0000","last_became_active":"2024-09-17T09:24:41.291705+0000","last_became_peered":"2024-09-17T09:24:41.291705+0000","last_unstale":"2024-09-17T09:24:48.467763+0000","last_undegraded":"2024-09-17T09:24:48.467763+0000","last_fullsized":"2024-09-17T09:24:48.467763+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:07:47.644642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"30'8","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T09:25:14.575425+0000","last_change":"2024-09-17T09:24:45.357226+0000","last_active":"2024-09-17T09:25:14.575425+0000","last_peered":"2024-09-17T09:25:14.575425+0000","last_clean":"2024-09-17T09:25:14.575425+0000","last_became_active":"2024-09-17T09:24:45.356987+0000","last_became_peered":"2024-09-17T09:24:45.356987+0000","last_unstale":"2024-09-17T09:25:14.575425+0000","last_undegraded":"2024-09-17T09:25:14.575425+0000","last_fullsized":"2024-09-17T09:25:14.575425+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:25:00.616606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355856+0000","last_change":"2024-09-17T09:24:47.362510+0000","last_active":"2024-09-17T09:24:48.355856+0000","last_peered":"2024-09-17T09:24:48.355856+0000","last_clean":"2024-09-17T09:24:48.355856+0000","last_became_active":"2024-09-17T09:24:47.362156+0000","last_became_peered":"2024-09-17T09:24:47.362156+0000","last_unstale":"2024-09-17T09:24:48.355856+0000","last_undegraded":"2024-09-17T09:24:48.355856+0000","last_fullsized":"2024-09-17T09:24:48.355856+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:55:57.173208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"30'11","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.670358+0000","last_change":"2024-09-17T09:24:43.341286+0000","last_active":"2024-09-17T09:24:48.670358+0000","last_peered":"2024-09-17T09:24:48.670358+0000","last_clean":"2024-09-17T09:24:48.670358+0000","last_became_active":"2024-09-17T09:24:43.340995+0000","last_became_peered":"2024-09-17T09:24:43.340995+0000","last_unstale":"2024-09-17T09:24:48.670358+0000","last_undegraded":"2024-09-17T09:24:48.670358+0000","last_fullsized":"2024-09-17T09:24:48.670358+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:13:44.266571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356140+0000","last_change":"2024-09-17T09:24:41.292908+0000","last_active":"2024-09-17T09:24:48.356140+0000","last_peered":"2024-09-17T09:24:48.356140+0000","last_clean":"2024-09-17T09:24:48.356140+0000","last_became_active":"2024-09-17T09:24:41.292128+0000","last_became_peered":"2024-09-17T09:24:41.292128+0000","last_unstale":"2024-09-17T09:24:48.356140+0000","last_undegraded":"2024-09-17T09:24:48.356140+0000","last_fullsized":"2024-09-17T09:24:48.356140+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:57.939876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357197+0000","last_change":"2024-09-17T09:24:45.358098+0000","last_active":"2024-09-17T09:24:48.357197+0000","last_peered":"2024-09-17T09:24:48.357197+0000","last_clean":"2024-09-17T09:24:48.357197+0000","last_became_active":"2024-09-17T09:24:45.357758+0000","last_became_peered":"2024-09-17T09:24:45.357758+0000","last_unstale":"2024-09-17T09:24:48.357197+0000","last_undegraded":"2024-09-17T09:24:48.357197+0000","last_fullsized":"2024-09-17T09:24:48.357197+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:16.380423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357147+0000","last_change":"2024-09-17T09:24:47.369920+0000","last_active":"2024-09-17T09:24:48.357147+0000","last_peered":"2024-09-17T09:24:48.357147+0000","last_clean":"2024-09-17T09:24:48.357147+0000","last_became_active":"2024-09-17T09:24:47.369616+0000","last_became_peered":"2024-09-17T09:24:47.369616+0000","last_unstale":"2024-09-17T09:24:48.357147+0000","last_undegraded":"2024-09-17T09:24:48.357147+0000","last_fullsized":"2024-09-17T09:24:48.357147+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:02.443979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"30'4","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.508265+0000","last_change":"2024-09-17T09:24:43.344364+0000","last_active":"2024-09-17T09:24:48.508265+0000","last_peered":"2024-09-17T09:24:48.508265+0000","last_clean":"2024-09-17T09:24:48.508265+0000","last_became_active":"2024-09-17T09:24:43.343962+0000","last_became_peered":"2024-09-17T09:24:43.343962+0000","last_unstale":"2024-09-17T09:24:48.508265+0000","last_undegraded":"2024-09-17T09:24:48.508265+0000","last_fullsized":"2024-09-17T09:24:48.508265+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:45:27.731863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.358780+0000","last_change":"2024-09-17T09:24:41.290247+0000","last_active":"2024-09-17T09:24:48.358780+0000","last_peered":"2024-09-17T09:24:48.358780+0000","last_clean":"2024-09-17T09:24:48.358780+0000","last_became_active":"2024-09-17T09:24:41.289562+0000","last_became_peered":"2024-09-17T09:24:41.289562+0000","last_unstale":"2024-09-17T09:24:48.358780+0000","last_undegraded":"2024-09-17T09:24:48.358780+0000","last_fullsized":"2024-09-17T09:24:48.358780+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:14.012718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357405+0000","last_change":"2024-09-17T09:24:45.357891+0000","last_active":"2024-09-17T09:24:48.357405+0000","last_peered":"2024-09-17T09:24:48.357405+0000","last_clean":"2024-09-17T09:24:48.357405+0000","last_became_active":"2024-09-17T09:24:45.357276+0000","last_became_peered":"2024-09-17T09:24:45.357276+0000","last_unstale":"2024-09-17T09:24:48.357405+0000","last_undegraded":"2024-09-17T09:24:48.357405+0000","last_fullsized":"2024-09-17T09:24:48.357405+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:42:38.055142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519370+0000","last_change":"2024-09-17T09:24:47.366422+0000","last_active":"2024-09-17T09:24:49.519370+0000","last_peered":"2024-09-17T09:24:49.519370+0000","last_clean":"2024-09-17T09:24:49.519370+0000","last_became_active":"2024-09-17T09:24:47.366097+0000","last_became_peered":"2024-09-17T09:24:47.366097+0000","last_unstale":"2024-09-17T09:24:49.519370+0000","last_undegraded":"2024-09-17T09:24:49.519370+0000","last_fullsized":"2024-09-17T09:24:49.519370+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:18:09.557860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"30'11","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.642997+0000","last_change":"2024-09-17T09:24:43.343012+0000","last_active":"2024-09-17T09:24:48.642997+0000","last_peered":"2024-09-17T09:24:48.642997+0000","last_clean":"2024-09-17T09:24:48.642997+0000","last_became_active":"2024-09-17T09:24:43.342644+0000","last_became_peered":"2024-09-17T09:24:43.342644+0000","last_unstale":"2024-09-17T09:24:48.642997+0000","last_undegraded":"2024-09-17T09:24:48.642997+0000","last_fullsized":"2024-09-17T09:24:48.642997+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:04.890408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356357+0000","last_change":"2024-09-17T09:24:41.291746+0000","last_active":"2024-09-17T09:24:48.356357+0000","last_peered":"2024-09-17T09:24:48.356357+0000","last_clean":"2024-09-17T09:24:48.356357+0000","last_became_active":"2024-09-17T09:24:41.290651+0000","last_became_peered":"2024-09-17T09:24:41.290651+0000","last_unstale":"2024-09-17T09:24:48.356357+0000","last_undegraded":"2024-09-17T09:24:48.356357+0000","last_fullsized":"2024-09-17T09:24:48.356357+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:30.406366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"30'8","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T09:25:14.575409+0000","last_change":"2024-09-17T09:24:45.357783+0000","last_active":"2024-09-17T09:25:14.575409+0000","last_peered":"2024-09-17T09:25:14.575409+0000","last_clean":"2024-09-17T09:25:14.575409+0000","last_became_active":"2024-09-17T09:24:45.356704+0000","last_became_peered":"2024-09-17T09:24:45.356704+0000","last_unstale":"2024-09-17T09:25:14.575409+0000","last_undegraded":"2024-09-17T09:25:14.575409+0000","last_fullsized":"2024-09-17T09:25:14.575409+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:04:35.792706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519343+0000","last_change":"2024-09-17T09:24:47.365791+0000","last_active":"2024-09-17T09:24:49.519343+0000","last_peered":"2024-09-17T09:24:49.519343+0000","last_clean":"2024-09-17T09:24:49.519343+0000","last_became_active":"2024-09-17T09:24:47.364730+0000","last_became_peered":"2024-09-17T09:24:47.364730+0000","last_unstale":"2024-09-17T09:24:49.519343+0000","last_undegraded":"2024-09-17T09:24:49.519343+0000","last_fullsized":"2024-09-17T09:24:49.519343+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:50:10.156618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.578853+0000","last_change":"2024-09-17T09:24:43.340422+0000","last_active":"2024-09-17T09:24:48.578853+0000","last_peered":"2024-09-17T09:24:48.578853+0000","last_clean":"2024-09-17T09:24:48.578853+0000","last_became_active":"2024-09-17T09:24:43.340091+0000","last_became_peered":"2024-09-17T09:24:43.340091+0000","last_unstale":"2024-09-17T09:24:48.578853+0000","last_undegraded":"2024-09-17T09:24:48.578853+0000","last_fullsized":"2024-09-17T09:24:48.578853+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:43.419138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356535+0000","last_change":"2024-09-17T09:24:41.291627+0000","last_active":"2024-09-17T09:24:48.356535+0000","last_peered":"2024-09-17T09:24:48.356535+0000","last_clean":"2024-09-17T09:24:48.356535+0000","last_became_active":"2024-09-17T09:24:41.291125+0000","last_became_peered":"2024-09-17T09:24:41.291125+0000","last_unstale":"2024-09-17T09:24:48.356535+0000","last_undegraded":"2024-09-17T09:24:48.356535+0000","last_fullsized":"2024-09-17T09:24:48.356535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:45:02.471463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"30'8","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T09:25:14.575342+0000","last_change":"2024-09-17T09:24:45.356466+0000","last_active":"2024-09-17T09:25:14.575342+0000","last_peered":"2024-09-17T09:25:14.575342+0000","last_clean":"2024-09-17T09:25:14.575342+0000","last_became_active":"2024-09-17T09:24:45.356202+0000","last_became_peered":"2024-09-17T09:24:45.356202+0000","last_unstale":"2024-09-17T09:25:14.575342+0000","last_undegraded":"2024-09-17T09:25:14.575342+0000","last_fullsized":"2024-09-17T09:25:14.575342+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:27:32.821368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355928+0000","last_change":"2024-09-17T09:24:47.366655+0000","last_active":"2024-09-17T09:24:48.355928+0000","last_peered":"2024-09-17T09:24:48.355928+0000","last_clean":"2024-09-17T09:24:48.355928+0000","last_became_active":"2024-09-17T09:24:47.366296+0000","last_became_peered":"2024-09-17T09:24:47.366296+0000","last_unstale":"2024-09-17T09:24:48.355928+0000","last_undegraded":"2024-09-17T09:24:48.355928+0000","last_fullsized":"2024-09-17T09:24:48.355928+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:26:10.401509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.653495+0000","last_change":"2024-09-17T09:24:43.341039+0000","last_active":"2024-09-17T09:24:48.653495+0000","last_peered":"2024-09-17T09:24:48.653495+0000","last_clean":"2024-09-17T09:24:48.653495+0000","last_became_active":"2024-09-17T09:24:43.340632+0000","last_became_peered":"2024-09-17T09:24:43.340632+0000","last_unstale":"2024-09-17T09:24:48.653495+0000","last_undegraded":"2024-09-17T09:24:48.653495+0000","last_fullsized":"2024-09-17T09:24:48.653495+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:30.610201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.14","version":"23'1","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.468117+0000","last_change":"2024-09-17T09:24:41.289714+0000","last_active":"2024-09-17T09:24:48.468117+0000","last_peered":"2024-09-17T09:24:48.468117+0000","last_clean":"2024-09-17T09:24:48.468117+0000","last_became_active":"2024-09-17T09:24:41.289175+0000","last_became_peered":"2024-09-17T09:24:41.289175+0000","last_unstale":"2024-09-17T09:24:48.468117+0000","last_undegraded":"2024-09-17T09:24:48.468117+0000","last_fullsized":"2024-09-17T09:24:48.468117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:37.018855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355979+0000","last_change":"2024-09-17T09:24:45.356905+0000","last_active":"2024-09-17T09:24:48.355979+0000","last_peered":"2024-09-17T09:24:48.355979+0000","last_clean":"2024-09-17T09:24:48.355979+0000","last_became_active":"2024-09-17T09:24:45.355852+0000","last_became_peered":"2024-09-17T09:24:45.355852+0000","last_unstale":"2024-09-17T09:24:48.355979+0000","last_undegraded":"2024-09-17T09:24:48.355979+0000","last_fullsized":"2024-09-17T09:24:48.355979+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:05:34.125013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356783+0000","last_change":"2024-09-17T09:24:47.362325+0000","last_active":"2024-09-17T09:24:48.356783+0000","last_peered":"2024-09-17T09:24:48.356783+0000","last_clean":"2024-09-17T09:24:48.356783+0000","last_became_active":"2024-09-17T09:24:47.362038+0000","last_became_peered":"2024-09-17T09:24:47.362038+0000","last_unstale":"2024-09-17T09:24:48.356783+0000","last_undegraded":"2024-09-17T09:24:48.356783+0000","last_fullsized":"2024-09-17T09:24:48.356783+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:56.853743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"30'10","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.651445+0000","last_change":"2024-09-17T09:24:43.343497+0000","last_active":"2024-09-17T09:24:48.651445+0000","last_peered":"2024-09-17T09:24:48.651445+0000","last_clean":"2024-09-17T09:24:48.651445+0000","last_became_active":"2024-09-17T09:24:43.343187+0000","last_became_peered":"2024-09-17T09:24:43.343187+0000","last_unstale":"2024-09-17T09:24:48.651445+0000","last_undegraded":"2024-09-17T09:24:48.651445+0000","last_fullsized":"2024-09-17T09:24:48.651445+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:16.265795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357663+0000","last_change":"2024-09-17T09:24:41.292441+0000","last_active":"2024-09-17T09:24:48.357663+0000","last_peered":"2024-09-17T09:24:48.357663+0000","last_clean":"2024-09-17T09:24:48.357663+0000","last_became_active":"2024-09-17T09:24:41.291371+0000","last_became_peered":"2024-09-17T09:24:41.291371+0000","last_unstale":"2024-09-17T09:24:48.357663+0000","last_undegraded":"2024-09-17T09:24:48.357663+0000","last_fullsized":"2024-09-17T09:24:48.357663+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:23:53.926344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.360298+0000","last_change":"2024-09-17T09:24:45.358331+0000","last_active":"2024-09-17T09:24:48.360298+0000","last_peered":"2024-09-17T09:24:48.360298+0000","last_clean":"2024-09-17T09:24:48.360298+0000","last_became_active":"2024-09-17T09:24:45.356223+0000","last_became_peered":"2024-09-17T09:24:45.356223+0000","last_unstale":"2024-09-17T09:24:48.360298+0000","last_undegraded":"2024-09-17T09:24:48.360298+0000","last_fullsized":"2024-09-17T09:24:48.360298+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:20:17.662397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357354+0000","last_change":"2024-09-17T09:24:47.363473+0000","last_active":"2024-09-17T09:24:48.357354+0000","last_peered":"2024-09-17T09:24:48.357354+0000","last_clean":"2024-09-17T09:24:48.357354+0000","last_became_active":"2024-09-17T09:24:47.363258+0000","last_became_peered":"2024-09-17T09:24:47.363258+0000","last_unstale":"2024-09-17T09:24:48.357354+0000","last_undegraded":"2024-09-17T09:24:48.357354+0000","last_fullsized":"2024-09-17T09:24:48.357354+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:28:11.055372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"30'6","reported_seq":25,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.635027+0000","last_change":"2024-09-17T09:24:43.342800+0000","last_active":"2024-09-17T09:24:48.635027+0000","last_peered":"2024-09-17T09:24:48.635027+0000","last_clean":"2024-09-17T09:24:48.635027+0000","last_became_active":"2024-09-17T09:24:43.341956+0000","last_became_peered":"2024-09-17T09:24:43.341956+0000","last_unstale":"2024-09-17T09:24:48.635027+0000","last_undegraded":"2024-09-17T09:24:48.635027+0000","last_fullsized":"2024-09-17T09:24:48.635027+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:23:43.435775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.360187+0000","last_change":"2024-09-17T09:24:41.290373+0000","last_active":"2024-09-17T09:24:48.360187+0000","last_peered":"2024-09-17T09:24:48.360187+0000","last_clean":"2024-09-17T09:24:48.360187+0000","last_became_active":"2024-09-17T09:24:41.289397+0000","last_became_peered":"2024-09-17T09:24:41.289397+0000","last_unstale":"2024-09-17T09:24:48.360187+0000","last_undegraded":"2024-09-17T09:24:48.360187+0000","last_fullsized":"2024-09-17T09:24:48.360187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:31.436622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355792+0000","last_change":"2024-09-17T09:24:45.360304+0000","last_active":"2024-09-17T09:24:48.355792+0000","last_peered":"2024-09-17T09:24:48.355792+0000","last_clean":"2024-09-17T09:24:48.355792+0000","last_became_active":"2024-09-17T09:24:45.359935+0000","last_became_peered":"2024-09-17T09:24:45.359935+0000","last_unstale":"2024-09-17T09:24:48.355792+0000","last_undegraded":"2024-09-17T09:24:48.355792+0000","last_fullsized":"2024-09-17T09:24:48.355792+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:03:34.460254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519146+0000","last_change":"2024-09-17T09:24:47.365029+0000","last_active":"2024-09-17T09:24:49.519146+0000","last_peered":"2024-09-17T09:24:49.519146+0000","last_clean":"2024-09-17T09:24:49.519146+0000","last_became_active":"2024-09-17T09:24:47.364460+0000","last_became_peered":"2024-09-17T09:24:47.364460+0000","last_unstale":"2024-09-17T09:24:49.519146+0000","last_undegraded":"2024-09-17T09:24:49.519146+0000","last_fullsized":"2024-09-17T09:24:49.519146+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:22.955015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.518888+0000","last_change":"2024-09-17T09:24:43.338583+0000","last_active":"2024-09-17T09:24:48.518888+0000","last_peered":"2024-09-17T09:24:48.518888+0000","last_clean":"2024-09-17T09:24:48.518888+0000","last_became_active":"2024-09-17T09:24:43.337983+0000","last_became_peered":"2024-09-17T09:24:43.337983+0000","last_unstale":"2024-09-17T09:24:48.518888+0000","last_undegraded":"2024-09-17T09:24:48.518888+0000","last_fullsized":"2024-09-17T09:24:48.518888+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:43.090170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.360244+0000","last_change":"2024-09-17T09:24:41.290274+0000","last_active":"2024-09-17T09:24:48.360244+0000","last_peered":"2024-09-17T09:24:48.360244+0000","last_clean":"2024-09-17T09:24:48.360244+0000","last_became_active":"2024-09-17T09:24:41.289094+0000","last_became_peered":"2024-09-17T09:24:41.289094+0000","last_unstale":"2024-09-17T09:24:48.360244+0000","last_undegraded":"2024-09-17T09:24:48.360244+0000","last_fullsized":"2024-09-17T09:24:48.360244+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:07:06.789137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355930+0000","last_change":"2024-09-17T09:24:45.358676+0000","last_active":"2024-09-17T09:24:48.355930+0000","last_peered":"2024-09-17T09:24:48.355930+0000","last_clean":"2024-09-17T09:24:48.355930+0000","last_became_active":"2024-09-17T09:24:45.358193+0000","last_became_peered":"2024-09-17T09:24:45.358193+0000","last_unstale":"2024-09-17T09:24:48.355930+0000","last_undegraded":"2024-09-17T09:24:48.355930+0000","last_fullsized":"2024-09-17T09:24:48.355930+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:48:40.388371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519099+0000","last_change":"2024-09-17T09:24:47.364713+0000","last_active":"2024-09-17T09:24:49.519099+0000","last_peered":"2024-09-17T09:24:49.519099+0000","last_clean":"2024-09-17T09:24:49.519099+0000","last_became_active":"2024-09-17T09:24:47.364345+0000","last_became_peered":"2024-09-17T09:24:47.364345+0000","last_unstale":"2024-09-17T09:24:49.519099+0000","last_undegraded":"2024-09-17T09:24:49.519099+0000","last_fullsized":"2024-09-17T09:24:49.519099+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:34:57.821269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519378+0000","last_change":"2024-09-17T09:24:47.366431+0000","last_active":"2024-09-17T09:24:49.519378+0000","last_peered":"2024-09-17T09:24:49.519378+0000","last_clean":"2024-09-17T09:24:49.519378+0000","last_became_active":"2024-09-17T09:24:47.364476+0000","last_became_peered":"2024-09-17T09:24:47.364476+0000","last_unstale":"2024-09-17T09:24:49.519378+0000","last_undegraded":"2024-09-17T09:24:49.519378+0000","last_fullsized":"2024-09-17T09:24:49.519378+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:02.430539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"30'15","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.676343+0000","last_change":"2024-09-17T09:24:43.341500+0000","last_active":"2024-09-17T09:24:48.676343+0000","last_peered":"2024-09-17T09:24:48.676343+0000","last_clean":"2024-09-17T09:24:48.676343+0000","last_became_active":"2024-09-17T09:24:43.341185+0000","last_became_peered":"2024-09-17T09:24:43.341185+0000","last_unstale":"2024-09-17T09:24:48.676343+0000","last_undegraded":"2024-09-17T09:24:48.676343+0000","last_fullsized":"2024-09-17T09:24:48.676343+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:11.317662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356243+0000","last_change":"2024-09-17T09:24:41.292712+0000","last_active":"2024-09-17T09:24:48.356243+0000","last_peered":"2024-09-17T09:24:48.356243+0000","last_clean":"2024-09-17T09:24:48.356243+0000","last_became_active":"2024-09-17T09:24:41.291236+0000","last_became_peered":"2024-09-17T09:24:41.291236+0000","last_unstale":"2024-09-17T09:24:48.356243+0000","last_undegraded":"2024-09-17T09:24:48.356243+0000","last_fullsized":"2024-09-17T09:24:48.356243+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:42.482034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357330+0000","last_change":"2024-09-17T09:24:45.358111+0000","last_active":"2024-09-17T09:24:48.357330+0000","last_peered":"2024-09-17T09:24:48.357330+0000","last_clean":"2024-09-17T09:24:48.357330+0000","last_became_active":"2024-09-17T09:24:45.357772+0000","last_became_peered":"2024-09-17T09:24:45.357772+0000","last_unstale":"2024-09-17T09:24:48.357330+0000","last_undegraded":"2024-09-17T09:24:48.357330+0000","last_fullsized":"2024-09-17T09:24:48.357330+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:59:23.575217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.356333+0000","last_change":"2024-09-17T09:24:47.368096+0000","last_active":"2024-09-17T09:24:48.356333+0000","last_peered":"2024-09-17T09:24:48.356333+0000","last_clean":"2024-09-17T09:24:48.356333+0000","last_became_active":"2024-09-17T09:24:47.367793+0000","last_became_peered":"2024-09-17T09:24:47.367793+0000","last_unstale":"2024-09-17T09:24:48.356333+0000","last_undegraded":"2024-09-17T09:24:48.356333+0000","last_fullsized":"2024-09-17T09:24:48.356333+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:15:08.514704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.488166+0000","last_change":"2024-09-17T09:24:43.342750+0000","last_active":"2024-09-17T09:24:48.488166+0000","last_peered":"2024-09-17T09:24:48.488166+0000","last_clean":"2024-09-17T09:24:48.488166+0000","last_became_active":"2024-09-17T09:24:43.342426+0000","last_became_peered":"2024-09-17T09:24:43.342426+0000","last_unstale":"2024-09-17T09:24:48.488166+0000","last_undegraded":"2024-09-17T09:24:48.488166+0000","last_fullsized":"2024-09-17T09:24:48.488166+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:13:56.840742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355268+0000","last_change":"2024-09-17T09:24:41.289873+0000","last_active":"2024-09-17T09:24:48.355268+0000","last_peered":"2024-09-17T09:24:48.355268+0000","last_clean":"2024-09-17T09:24:48.355268+0000","last_became_active":"2024-09-17T09:24:41.289429+0000","last_became_peered":"2024-09-17T09:24:41.289429+0000","last_unstale":"2024-09-17T09:24:48.355268+0000","last_undegraded":"2024-09-17T09:24:48.355268+0000","last_fullsized":"2024-09-17T09:24:48.355268+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:00.457411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T09:25:14.575360+0000","last_change":"2024-09-17T09:24:45.356796+0000","last_active":"2024-09-17T09:25:14.575360+0000","last_peered":"2024-09-17T09:25:14.575360+0000","last_clean":"2024-09-17T09:25:14.575360+0000","last_became_active":"2024-09-17T09:24:45.356524+0000","last_became_peered":"2024-09-17T09:24:45.356524+0000","last_unstale":"2024-09-17T09:25:14.575360+0000","last_undegraded":"2024-09-17T09:25:14.575360+0000","last_fullsized":"2024-09-17T09:25:14.575360+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:11:50.229474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.358687+0000","last_change":"2024-09-17T09:24:41.288858+0000","last_active":"2024-09-17T09:24:48.358687+0000","last_peered":"2024-09-17T09:24:48.358687+0000","last_clean":"2024-09-17T09:24:48.358687+0000","last_became_active":"2024-09-17T09:24:41.288561+0000","last_became_peered":"2024-09-17T09:24:41.288561+0000","last_unstale":"2024-09-17T09:24:48.358687+0000","last_undegraded":"2024-09-17T09:24:48.358687+0000","last_fullsized":"2024-09-17T09:24:48.358687+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:00.666989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"30'5","reported_seq":26,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.565069+0000","last_change":"2024-09-17T09:24:43.340901+0000","last_active":"2024-09-17T09:24:48.565069+0000","last_peered":"2024-09-17T09:24:48.565069+0000","last_clean":"2024-09-17T09:24:48.565069+0000","last_became_active":"2024-09-17T09:24:43.340593+0000","last_became_peered":"2024-09-17T09:24:43.340593+0000","last_unstale":"2024-09-17T09:24:48.565069+0000","last_undegraded":"2024-09-17T09:24:48.565069+0000","last_fullsized":"2024-09-17T09:24:48.565069+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:23.274367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.357500+0000","last_change":"2024-09-17T09:24:45.357747+0000","last_active":"2024-09-17T09:24:48.357500+0000","last_peered":"2024-09-17T09:24:48.357500+0000","last_clean":"2024-09-17T09:24:48.357500+0000","last_became_active":"2024-09-17T09:24:45.356367+0000","last_became_peered":"2024-09-17T09:24:45.356367+0000","last_unstale":"2024-09-17T09:24:48.357500+0000","last_undegraded":"2024-09-17T09:24:48.357500+0000","last_fullsized":"2024-09-17T09:24:48.357500+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:17.465071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.355713+0000","last_change":"2024-09-17T09:24:47.368183+0000","last_active":"2024-09-17T09:24:48.355713+0000","last_peered":"2024-09-17T09:24:48.355713+0000","last_clean":"2024-09-17T09:24:48.355713+0000","last_became_active":"2024-09-17T09:24:47.367876+0000","last_became_peered":"2024-09-17T09:24:47.367876+0000","last_unstale":"2024-09-17T09:24:48.355713+0000","last_undegraded":"2024-09-17T09:24:48.355713+0000","last_fullsized":"2024-09-17T09:24:48.355713+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:41.867731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.456917+0000","last_change":"2024-09-17T09:24:41.291466+0000","last_active":"2024-09-17T09:24:48.456917+0000","last_peered":"2024-09-17T09:24:48.456917+0000","last_clean":"2024-09-17T09:24:48.456917+0000","last_became_active":"2024-09-17T09:24:41.290806+0000","last_became_peered":"2024-09-17T09:24:41.290806+0000","last_unstale":"2024-09-17T09:24:48.456917+0000","last_undegraded":"2024-09-17T09:24:48.456917+0000","last_fullsized":"2024-09-17T09:24:48.456917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:40.269994+0000","last_clean_scrub_stamp":"2024-09-17T09:24:40.269994+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:08.618737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"30'9","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.585319+0000","last_change":"2024-09-17T09:24:43.340573+0000","last_active":"2024-09-17T09:24:48.585319+0000","last_peered":"2024-09-17T09:24:48.585319+0000","last_clean":"2024-09-17T09:24:48.585319+0000","last_became_active":"2024-09-17T09:24:43.340285+0000","last_became_peered":"2024-09-17T09:24:43.340285+0000","last_unstale":"2024-09-17T09:24:48.585319+0000","last_undegraded":"2024-09-17T09:24:48.585319+0000","last_fullsized":"2024-09-17T09:24:48.585319+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:42.324190+0000","last_clean_scrub_stamp":"2024-09-17T09:24:42.324190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:50.874938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:48.358122+0000","last_change":"2024-09-17T09:24:45.360363+0000","last_active":"2024-09-17T09:24:48.358122+0000","last_peered":"2024-09-17T09:24:48.358122+0000","last_clean":"2024-09-17T09:24:48.358122+0000","last_became_active":"2024-09-17T09:24:45.360049+0000","last_became_peered":"2024-09-17T09:24:45.360049+0000","last_unstale":"2024-09-17T09:24:48.358122+0000","last_undegraded":"2024-09-17T09:24:48.358122+0000","last_fullsized":"2024-09-17T09:24:48.358122+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:44.335894+0000","last_clean_scrub_stamp":"2024-09-17T09:24:44.335894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:17.291481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T09:24:49.519305+0000","last_change":"2024-09-17T09:24:47.365034+0000","last_active":"2024-09-17T09:24:49.519305+0000","last_peered":"2024-09-17T09:24:49.519305+0000","last_clean":"2024-09-17T09:24:49.519305+0000","last_became_active":"2024-09-17T09:24:47.364461+0000","last_became_peered":"2024-09-17T09:24:47.364461+0000","last_unstale":"2024-09-17T09:24:49.519305+0000","last_undegraded":"2024-09-17T09:24:49.519305+0000","last_fullsized":"2024-09-17T09:24:49.519305+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:24:46.345391+0000","last_clean_scrub_stamp":"2024-09-17T09:24:46.345391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:43.160937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":18,"seq":77309411342,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32132,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708924,"statfs":{"total":95990841344,"available":95957938176,"internally_reserved":0,"allocated":1040384,"data_stored":699308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32136,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708920,"statfs":{"total":95990841344,"available":95957934080,"internally_reserved":0,"allocated":1044480,"data_stored":699977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 09:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738390,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32132,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708924,"statfs":{"total":95990841344,"available":95957938176,"internally_reserved":0,"allocated":1044480,"data_stored":699977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 09:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T09:25:28.902 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-17T09:25:28.902 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T09:25:28.903 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-17T09:25:28.903 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-17T09:25:29.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:28 smithi077 ceph-mon[30659]: from='client.14529 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T09:25:29.063 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:28 smithi183 ceph-mon[30372]: from='client.14529 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T09:25:29.202 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:25:29 smithi003 podman[55971]: 2024-09-17 09:25:29.201714916 +0000 UTC m=+0.207058738 container create 3379156a6600b24928ac927cb9171e23988727cf28539aa9b61e77d334851c7b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T09:25:29.631 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:25:29.744 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: [17/Sep/2024:09:25:29] ENGINE Bus STOPPING 2024-09-17T09:25:29.745 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:25:29 smithi003 podman[55971]: 2024-09-17 09:25:29.487489266 +0000 UTC m=+0.492833054 container init 3379156a6600b24928ac927cb9171e23988727cf28539aa9b61e77d334851c7b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T09:25:29.745 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:25:29 smithi003 podman[55971]: 2024-09-17 09:25:29.492766148 +0000 UTC m=+0.498109927 container start 3379156a6600b24928ac927cb9171e23988727cf28539aa9b61e77d334851c7b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T09:25:29.745 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-alertmanager-a[55997]: ts=2024-09-17T09:25:29.505Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-17T09:25:29.745 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-alertmanager-a[55997]: ts=2024-09-17T09:25:29.505Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-17T09:25:29.745 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-alertmanager-a[55997]: ts=2024-09-17T09:25:29.506Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.3 port=9094 2024-09-17T09:25:29.746 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-alertmanager-a[55997]: ts=2024-09-17T09:25:29.507Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-17T09:25:29.746 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-alertmanager-a[55997]: ts=2024-09-17T09:25:29.530Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-17T09:25:29.746 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-alertmanager-a[55997]: ts=2024-09-17T09:25:29.530Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-17T09:25:29.746 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-alertmanager-a[55997]: ts=2024-09-17T09:25:29.533Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-17T09:25:29.746 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-alertmanager-a[55997]: ts=2024-09-17T09:25:29.533Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-17T09:25:29.746 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:25:29 smithi003 bash[55971]: 3379156a6600b24928ac927cb9171e23988727cf28539aa9b61e77d334851c7b 2024-09-17T09:25:29.746 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:25:29 smithi003 systemd[1]: Started Ceph alertmanager.a for 08f126da-74d6-11ef-bceb-c7b262605968. 2024-09-17T09:25:29.997 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:29 smithi077 ceph-mon[30659]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:29.997 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:29 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:29.997 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:29 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:29.997 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:29 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:29.997 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:29 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:29.997 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:29 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:29.997 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:29 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:29.997 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:29 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T09:25:29.998 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:29 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:30.008 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: [17/Sep/2024:09:25:29] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T09:25:30.008 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: [17/Sep/2024:09:25:29] ENGINE Bus STOPPED 2024-09-17T09:25:30.008 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: [17/Sep/2024:09:25:29] ENGINE Bus STARTING 2024-09-17T09:25:30.008 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: [17/Sep/2024:09:25:29] ENGINE Serving on http://:::9283 2024-09-17T09:25:30.009 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: [17/Sep/2024:09:25:29] ENGINE Bus STARTED 2024-09-17T09:25:30.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-mon[25908]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:30.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:30.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:30.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:30.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:30.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:30.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:30.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T09:25:30.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:29 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:30.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:29 smithi183 ceph-mon[30372]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:30.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:29 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:30.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:29 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:30.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:29 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:30.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:29 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:30.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:29 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:30.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:29 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:30.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:29 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T09:25:30.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:29 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:30.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:30 smithi183 ceph-mon[30372]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-17T09:25:30.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:30 smithi183 ceph-mon[30372]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T09:25:30.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:30 smithi183 ceph-mon[30372]: Deploying daemon grafana.a on smithi077 2024-09-17T09:25:30.836 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:30 smithi003 ceph-mon[25908]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-17T09:25:30.836 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:30 smithi003 ceph-mon[25908]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T09:25:30.837 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:30 smithi003 ceph-mon[25908]: Deploying daemon grafana.a on smithi077 2024-09-17T09:25:31.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:30 smithi077 ceph-mon[30659]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-17T09:25:31.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:30 smithi077 ceph-mon[30659]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T09:25:31.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:30 smithi077 ceph-mon[30659]: Deploying daemon grafana.a on smithi077 2024-09-17T09:25:31.519 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-17T09:25:31.519 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T09:25:32.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:31 smithi077 ceph-mon[30659]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:32.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:31 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:32.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:31 smithi077 ceph-mon[30659]: from='client.? 172.21.15.3:0/39254898' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-17T09:25:32.008 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:25:31 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-alertmanager-a[55997]: ts=2024-09-17T09:25:31.507Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000536753s 2024-09-17T09:25:32.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:31 smithi003 ceph-mon[25908]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:32.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:31 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:32.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:31 smithi003 ceph-mon[25908]: from='client.? 172.21.15.3:0/39254898' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-17T09:25:32.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:31 smithi183 ceph-mon[30372]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:32.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:31 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:32.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:31 smithi183 ceph-mon[30372]: from='client.? 172.21.15.3:0/39254898' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-17T09:25:32.378 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-17T09:25:32.379 INFO:tasks.cephadm:Setup complete, yielding 2024-09-17T09:25:32.379 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-17T09:25:32.391 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-09-17T09:25:32.392 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- bash -c 'stat -c '"'"'%u %g'"'"' /var/log/ceph | grep '"'"'167 167'"'"'' 2024-09-17T09:25:32.598 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:25:34.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:33 smithi003 ceph-mon[25908]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:34.008 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:33 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: ::ffff:172.21.15.77 - - [17/Sep/2024:09:25:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T09:25:34.063 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:33 smithi183 ceph-mon[30372]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:34.235 INFO:teuthology.orchestra.run.smithi003.stdout:167 167 2024-09-17T09:25:34.251 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:33 smithi077 ceph-mon[30659]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:34.874 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-17T09:25:35.081 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:25:36.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:35 smithi003 ceph-mon[25908]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:36.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:35 smithi183 ceph-mon[30372]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:36.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:35 smithi077 ceph-mon[30659]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:36.896 INFO:teuthology.orchestra.run.smithi003.stdout:Backend: cephadm 2024-09-17T09:25:36.896 INFO:teuthology.orchestra.run.smithi003.stdout:Available: Yes 2024-09-17T09:25:36.896 INFO:teuthology.orchestra.run.smithi003.stdout:Paused: No 2024-09-17T09:25:37.025 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:36 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:25:37.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:36 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:25:37.200 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:36 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:25:37.522 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-17T09:25:37.720 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:25:38.013 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:37 smithi003 ceph-mon[25908]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:38.013 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:37 smithi003 ceph-mon[25908]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:38.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:37 smithi183 ceph-mon[30372]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:38.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:37 smithi183 ceph-mon[30372]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:38.089 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:37 smithi077 ceph-mon[30659]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:38.089 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:37 smithi077 ceph-mon[30659]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:39.522 INFO:teuthology.orchestra.run.smithi003.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T09:25:39.522 INFO:teuthology.orchestra.run.smithi003.stdout:alertmanager.a smithi003 *:9093,9094 starting - - - - 2024-09-17T09:25:39.522 INFO:teuthology.orchestra.run.smithi003.stdout:mgr.x smithi077 *:8443,8765 running (2m) 30s ago 2m 436M - 18.2.4-855-g19e2a2be 980a0bfcd684 0e9c55eeabe0 2024-09-17T09:25:39.522 INFO:teuthology.orchestra.run.smithi003.stdout:mgr.y smithi003 *:9283,8765 running (4m) 25s ago 4m 466M - 18.2.4-855-g19e2a2be 980a0bfcd684 e24d87b96bc5 2024-09-17T09:25:39.522 INFO:teuthology.orchestra.run.smithi003.stdout:mgr.z smithi183 *:8443,8765 running (2m) 31s ago 2m 436M - 18.2.4-855-g19e2a2be 980a0bfcd684 18462c45a5cb 2024-09-17T09:25:39.523 INFO:teuthology.orchestra.run.smithi003.stdout:mon.a smithi003 running (4m) 25s ago 4m 48.7M 2048M 18.2.4-855-g19e2a2be 980a0bfcd684 9b15ead85423 2024-09-17T09:25:39.523 INFO:teuthology.orchestra.run.smithi003.stdout:mon.b smithi077 running (2m) 30s ago 2m 40.9M 2048M 18.2.4-855-g19e2a2be 980a0bfcd684 2f9641ab2c3a 2024-09-17T09:25:39.523 INFO:teuthology.orchestra.run.smithi003.stdout:mon.c smithi183 running (2m) 31s ago 2m 43.5M 2048M 18.2.4-855-g19e2a2be 980a0bfcd684 d00a181b8641 2024-09-17T09:25:39.523 INFO:teuthology.orchestra.run.smithi003.stdout:osd.0 smithi003 running (2m) 25s ago 2m 50.5M 15.6G 18.2.4-855-g19e2a2be 980a0bfcd684 65078fd726b4 2024-09-17T09:25:39.523 INFO:teuthology.orchestra.run.smithi003.stdout:osd.1 smithi077 running (102s) 30s ago 102s 49.8M 15.6G 18.2.4-855-g19e2a2be 980a0bfcd684 982688de797b 2024-09-17T09:25:39.523 INFO:teuthology.orchestra.run.smithi003.stdout:osd.2 smithi183 running (80s) 31s ago 80s 57.0M 16.6G 18.2.4-855-g19e2a2be 980a0bfcd684 0c45ea4ba1c6 2024-09-17T09:25:39.523 INFO:teuthology.orchestra.run.smithi003.stdout:prometheus.a smithi077 *:9095 running (40s) 30s ago 39s 25.9M - 2.43.0 a07b618ecd1d baa612d113ed 2024-09-17T09:25:39.523 INFO:teuthology.orchestra.run.smithi003.stdout:rgw.foo.a smithi003 *:80 running (60s) 25s ago 59s 74.4M - 18.2.4-855-g19e2a2be 980a0bfcd684 13fcd40fc00c 2024-09-17T09:25:39.752 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:39 smithi077 systemd[1]: Starting Ceph grafana.a for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:25:39.765 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:25:39 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-alertmanager-a[55997]: ts=2024-09-17T09:25:39.510Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002723276s 2024-09-17T09:25:40.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:39 smithi183 ceph-mon[30372]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:40.079 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:39 smithi003 ceph-mon[25908]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:40.101 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-17T09:25:40.123 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:39 smithi077 ceph-mon[30659]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:40.124 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:39 smithi077 podman[42764]: 2024-09-17 09:25:39.769910426 +0000 UTC m=+0.015315962 image pull 954c08fa618887fe9a6fe572ae541197c77e625b05f4193df7898f5ac649ada9 quay.io/ceph/ceph-grafana:9.4.7 2024-09-17T09:25:40.124 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:39 smithi077 podman[42764]: 2024-09-17 09:25:39.879658688 +0000 UTC m=+0.125064210 container create 23abc9d6b18fe0e2c1ed1d7b906b8fbfa5dff66d2e5e1fd29a337903fdb7b067 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a, description=Ceph Grafana Container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, io.buildah.version=1.32.0, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, architecture=x86_64, summary=Grafana Container configured for Ceph mgr/dashboard integration, maintainer=Paul Cuzner , vendor=Red Hat, Inc., io.openshift.tags=base rhel8, io.k8s.display-name=Red Hat Universal Base Image 8, distribution-scope=public, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=, com.redhat.component=ubi8-container, name=ubi8, vcs-type=git, release=239.1651231664) 2024-09-17T09:25:40.124 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 podman[42764]: 2024-09-17 09:25:40.123345628 +0000 UTC m=+0.368751164 container init 23abc9d6b18fe0e2c1ed1d7b906b8fbfa5dff66d2e5e1fd29a337903fdb7b067 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a, io.openshift.tags=base rhel8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., build-date=2022-04-29T11:48:11.055897, vcs-type=git, vendor=Red Hat, Inc., io.openshift.expose-services=, distribution-scope=public, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, description=Ceph Grafana Container, maintainer=Paul Cuzner , summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, version=8.5, name=ubi8, io.buildah.version=1.32.0, io.k8s.display-name=Red Hat Universal Base Image 8, architecture=x86_64, com.redhat.component=ubi8-container) 2024-09-17T09:25:40.301 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:25:40.375 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 podman[42764]: 2024-09-17 09:25:40.128086113 +0000 UTC m=+0.373491646 container start 23abc9d6b18fe0e2c1ed1d7b906b8fbfa5dff66d2e5e1fd29a337903fdb7b067 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a, name=ubi8, io.buildah.version=1.32.0, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, release=239.1651231664, vendor=Red Hat, Inc., io.openshift.tags=base rhel8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, maintainer=Paul Cuzner , io.k8s.display-name=Red Hat Universal Base Image 8, vcs-type=git, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.openshift.expose-services=, com.redhat.component=ubi8-container, architecture=x86_64, summary=Grafana Container configured for Ceph mgr/dashboard integration, description=Ceph Grafana Container, build-date=2022-04-29T11:48:11.055897, distribution-scope=public) 2024-09-17T09:25:40.375 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 bash[42764]: 23abc9d6b18fe0e2c1ed1d7b906b8fbfa5dff66d2e5e1fd29a337903fdb7b067 2024-09-17T09:25:40.376 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 systemd[1]: Started Ceph grafana.a for 08f126da-74d6-11ef-bceb-c7b262605968. 2024-09-17T09:25:40.376 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=settings t=2024-09-17T09:25:40.285558656Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-17T09:25:40.376 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=settings t=2024-09-17T09:25:40.285887667Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-17T09:25:40.376 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=settings t=2024-09-17T09:25:40.285908674Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-17T09:25:40.376 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=settings t=2024-09-17T09:25:40.285916929Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-17T09:25:40.376 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=settings t=2024-09-17T09:25:40.285923085Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-17T09:25:40.376 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=settings t=2024-09-17T09:25:40.285929258Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-17T09:25:40.376 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=settings t=2024-09-17T09:25:40.285935115Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-17T09:25:40.376 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=settings t=2024-09-17T09:25:40.285941079Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-17T09:25:40.376 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=settings t=2024-09-17T09:25:40.285947049Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-17T09:25:40.377 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=settings t=2024-09-17T09:25:40.285952954Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-17T09:25:40.377 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=settings t=2024-09-17T09:25:40.285958557Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-17T09:25:40.377 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=settings t=2024-09-17T09:25:40.28596408Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-17T09:25:40.377 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=settings t=2024-09-17T09:25:40.285969908Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-17T09:25:40.377 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=settings t=2024-09-17T09:25:40.285975956Z level=info msg="App mode production" 2024-09-17T09:25:40.377 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=sqlstore t=2024-09-17T09:25:40.286025286Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-17T09:25:40.377 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=sqlstore t=2024-09-17T09:25:40.286040892Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-17T09:25:40.377 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.28691236Z level=info msg="Starting DB migrations" 2024-09-17T09:25:40.377 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.286988234Z level=info msg="Executing migration" id="create migration_log table" 2024-09-17T09:25:40.377 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.287481648Z level=info msg="Executing migration" id="create user table" 2024-09-17T09:25:40.377 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.287907119Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-17T09:25:40.377 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.290662385Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-17T09:25:40.377 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.293280982Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-17T09:25:40.378 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.295938738Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-17T09:25:40.378 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.298707104Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-17T09:25:40.378 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.301406113Z level=info msg="Executing migration" id="create user table v2" 2024-09-17T09:25:40.378 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.304088377Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-17T09:25:40.378 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.306863785Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-17T09:25:40.378 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.309455271Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-17T09:25:40.378 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.312172034Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-17T09:25:40.378 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.312543806Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-17T09:25:40.378 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.315411159Z level=info msg="Executing migration" id="Update user table charset" 2024-09-17T09:25:40.379 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.318013798Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-17T09:25:40.379 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.320866301Z level=info msg="Executing migration" id="Add missing user data" 2024-09-17T09:25:40.379 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.321213694Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-17T09:25:40.380 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.322048802Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-17T09:25:40.380 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.322470892Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-17T09:25:40.380 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.322851031Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-17T09:25:40.380 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.323973723Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-17T09:25:40.380 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.324385143Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-17T09:25:40.380 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.324793598Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-17T09:25:40.381 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.325598132Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-17T09:25:40.381 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.326444882Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-17T09:25:40.381 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.327270283Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-17T09:25:40.381 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.328060768Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-17T09:25:40.381 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.328850632Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-17T09:25:40.381 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.329781647Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-17T09:25:40.381 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.330484299Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-17T09:25:40.381 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.331333569Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-17T09:25:40.382 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.33214721Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-17T09:25:40.382 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.332567419Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-17T09:25:40.382 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.333367176Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-17T09:25:40.382 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.334317796Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-17T09:25:40.382 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.335032465Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-17T09:25:40.382 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.335890937Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-17T09:25:40.382 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.336762459Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-17T09:25:40.382 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.337725382Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-17T09:25:40.382 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.338458801Z level=info msg="Executing migration" id="create star table" 2024-09-17T09:25:40.382 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.338809607Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-17T09:25:40.382 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.33922541Z level=info msg="Executing migration" id="create org table v1" 2024-09-17T09:25:40.382 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.339699752Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-17T09:25:40.382 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.3405571Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-17T09:25:40.382 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.340978477Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-17T09:25:40.383 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.341804574Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-17T09:25:40.383 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.342721047Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-17T09:25:40.383 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.343493314Z level=info msg="Executing migration" id="Update org table charset" 2024-09-17T09:25:40.383 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.343810985Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-17T09:25:40.383 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.344099941Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-17T09:25:40.383 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.344653518Z level=info msg="Executing migration" id="create dashboard table" 2024-09-17T09:25:40.383 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.345054545Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-17T09:25:40.383 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.345522571Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-17T09:25:40.383 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.348270991Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-17T09:25:40.383 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.348658655Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-17T09:25:40.383 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.35135293Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-17T09:25:40.383 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.354179325Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-17T09:25:40.383 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.356823908Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-17T09:25:40.384 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.357211972Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-17T09:25:40.384 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.360131174Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-17T09:25:40.384 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.362754491Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-17T09:25:40.384 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.365610876Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-17T09:25:40.384 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.36601941Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-17T09:25:40.384 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.368916216Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-17T09:25:40.384 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.37180689Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-17T09:25:40.384 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.374627269Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-17T09:25:40.384 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.37540864Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-17T09:25:40.626 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.377962632Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-17T09:25:40.626 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.380794014Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-17T09:25:40.626 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.381648554Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-17T09:25:40.627 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.382240814Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-17T09:25:40.627 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.383166829Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-17T09:25:40.627 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.383926008Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-17T09:25:40.627 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.384433625Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-17T09:25:40.627 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.3849392Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-17T09:25:40.627 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.385749675Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-17T09:25:40.627 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.386702189Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-17T09:25:40.627 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.38742867Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-17T09:25:40.627 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.388369196Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-17T09:25:40.627 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.389233429Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-17T09:25:40.627 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.389938703Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-17T09:25:40.627 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.39088784Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-17T09:25:40.627 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.391476708Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-17T09:25:40.628 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.392360129Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-17T09:25:40.628 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.393338134Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-17T09:25:40.629 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.394045551Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-17T09:25:40.629 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.394477429Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-17T09:25:40.629 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.395291091Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-17T09:25:40.629 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.395670828Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-17T09:25:40.629 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.396201212Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-17T09:25:40.629 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.397078668Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-17T09:25:40.629 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.397407731Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-17T09:25:40.629 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.398339575Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-17T09:25:40.630 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.39892487Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-17T09:25:40.630 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.39948625Z level=info msg="Executing migration" id="create data_source table" 2024-09-17T09:25:40.630 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.400352722Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-17T09:25:40.630 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.401144768Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-17T09:25:40.630 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.40156667Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-17T09:25:40.630 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.40240363Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-17T09:25:40.630 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.403286636Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-17T09:25:40.630 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.404526619Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-17T09:25:40.630 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.405217518Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-17T09:25:40.630 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.406173449Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-17T09:25:40.630 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.406918034Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-17T09:25:40.630 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.407289666Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-17T09:25:40.631 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.408135414Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-17T09:25:40.631 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.408702069Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-17T09:25:40.631 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.410875368Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-17T09:25:40.631 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.411184207Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-17T09:25:40.631 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.411669828Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-17T09:25:40.631 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.412023852Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-17T09:25:40.631 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.414400616Z level=info msg="Executing migration" id="Add uid column" 2024-09-17T09:25:40.631 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.414999205Z level=info msg="Executing migration" id="Update uid value" 2024-09-17T09:25:40.631 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.415337989Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-17T09:25:40.631 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.417759961Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-17T09:25:40.631 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.418363047Z level=info msg="Executing migration" id="create api_key table" 2024-09-17T09:25:40.631 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.420668653Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-17T09:25:40.631 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.423482145Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-17T09:25:40.631 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.426276618Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-17T09:25:40.632 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.429228538Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-17T09:25:40.632 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.432128132Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-17T09:25:40.632 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.432623691Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-17T09:25:40.632 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.433035859Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-17T09:25:40.632 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.434211785Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-17T09:25:40.632 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.434595086Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-17T09:25:40.632 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.435001109Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-17T09:25:40.632 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.435401589Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-17T09:25:40.632 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.435803739Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-17T09:25:40.632 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.436121046Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-17T09:25:40.632 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.436442479Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-17T09:25:40.632 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.436690434Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-17T09:25:40.633 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.437232665Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-17T09:25:40.633 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.437756104Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-17T09:25:40.633 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.438076336Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-17T09:25:40.633 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.438580843Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-17T09:25:40.633 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.439120687Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-17T09:25:40.633 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.439507806Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-17T09:25:40.633 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.439865759Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-17T09:25:40.633 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.440261865Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-17T09:25:40.633 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.440668745Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-17T09:25:40.633 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.441080263Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-17T09:25:40.633 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.441474491Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-17T09:25:40.633 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.441749593Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-17T09:25:40.633 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.442058634Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-17T09:25:40.634 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.442618423Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-17T09:25:40.634 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.443172161Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-17T09:25:40.634 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.443480235Z level=info msg="Executing migration" id="create quota table v1" 2024-09-17T09:25:40.634 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.443896591Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-17T09:25:40.634 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.444332242Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-17T09:25:40.634 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.444601415Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-17T09:25:40.634 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.445002171Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-17T09:25:40.634 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.4454213Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-17T09:25:40.634 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.445992145Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-17T09:25:40.634 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.4462887Z level=info msg="Executing migration" id="create session table" 2024-09-17T09:25:40.634 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.446702331Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-17T09:25:40.634 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.449469048Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-17T09:25:40.634 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.449897237Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-17T09:25:40.635 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.450316174Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-17T09:25:40.635 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.450701803Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-17T09:25:40.635 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.450994036Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-17T09:25:40.635 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.451286291Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-17T09:25:40.635 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.451558275Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-17T09:25:40.635 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.45183496Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-17T09:25:40.635 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.452223855Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-17T09:25:40.635 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.452484306Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-17T09:25:40.635 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.453067065Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-17T09:25:40.635 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.453378911Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-17T09:25:40.635 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.453985243Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-17T09:25:40.635 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.454564038Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-17T09:25:40.635 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.454844019Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-17T09:25:40.636 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.455283016Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-17T09:25:40.636 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.455740117Z level=info msg="Executing migration" id="create alert table v1" 2024-09-17T09:25:40.636 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.458563215Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-17T09:25:40.636 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.461685974Z level=info msg="Executing migration" id="add index alert state" 2024-09-17T09:25:40.636 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.464358393Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-17T09:25:40.636 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.464993682Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-17T09:25:40.636 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.465405086Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-17T09:25:40.636 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.465884766Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-17T09:25:40.636 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.466349263Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-17T09:25:40.636 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.469171117Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-17T09:25:40.636 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.469624137Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-17T09:25:40.636 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.47009577Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-17T09:25:40.637 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.4704143Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-17T09:25:40.637 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.470753947Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-17T09:25:40.637 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.472468976Z level=info msg="Executing migration" id="Add column is_default" 2024-09-17T09:25:40.637 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.475164566Z level=info msg="Executing migration" id="Add column frequency" 2024-09-17T09:25:40.637 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.475791819Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-17T09:25:40.637 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.476436085Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-17T09:25:40.637 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.477072852Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-17T09:25:40.637 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.477495575Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-17T09:25:40.637 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.477756945Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-17T09:25:40.637 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.478079237Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-17T09:25:40.637 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.47848735Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-17T09:25:40.637 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.478947172Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-17T09:25:40.638 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.479331318Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-17T09:25:40.638 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.482014294Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-17T09:25:40.638 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.482469483Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-17T09:25:40.640 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.484588441Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-17T09:25:40.640 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.487155081Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-17T09:25:40.640 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.489991837Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-17T09:25:40.640 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.492610758Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-17T09:25:40.640 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.495373364Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-17T09:25:40.640 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.498047384Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-17T09:25:40.640 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.500664015Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-17T09:25:40.640 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.503538438Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-17T09:25:40.640 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.505996427Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-17T09:25:40.641 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.508800128Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-17T09:25:40.641 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.511318381Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-17T09:25:40.641 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.513698981Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-17T09:25:40.641 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.516238162Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-17T09:25:40.641 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.518985475Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-17T09:25:40.641 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.521474064Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-17T09:25:40.641 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.524412024Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-17T09:25:40.641 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.524795776Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-17T09:25:40.641 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.527584884Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-17T09:25:40.641 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.529986494Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-17T09:25:40.641 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.532697683Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-17T09:25:40.641 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.535339085Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-17T09:25:40.641 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.537936824Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-17T09:25:40.642 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.540630695Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-17T09:25:40.642 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.543341641Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-17T09:25:40.642 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.546082788Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-17T09:25:40.642 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.548749309Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-17T09:25:40.643 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.551297557Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-17T09:25:40.643 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.554006083Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-17T09:25:40.643 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.556688689Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-17T09:25:40.643 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.559268619Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-17T09:25:40.643 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.560044802Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-17T09:25:40.643 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.560970361Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-17T09:25:40.643 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.561644542Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-17T09:25:40.643 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.562463068Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-17T09:25:40.647 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.563300295Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-17T09:25:40.647 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.564072735Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-17T09:25:40.647 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.564992544Z level=info msg="Executing migration" id="Move region to single row" 2024-09-17T09:25:40.648 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.565697709Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-17T09:25:40.648 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.566589766Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-17T09:25:40.648 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.56733412Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-17T09:25:40.648 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.568147123Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-17T09:25:40.648 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.568941965Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-17T09:25:40.649 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.56990801Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-17T09:25:40.649 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.570549611Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-17T09:25:40.649 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.571451216Z level=info msg="Executing migration" id="create test_data table" 2024-09-17T09:25:40.649 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.572084178Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-17T09:25:40.649 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.572507846Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-17T09:25:40.649 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.573368678Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-17T09:25:40.650 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.574330019Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-17T09:25:40.650 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.575072006Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-17T09:25:40.650 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.575951065Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-17T09:25:40.650 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.576932279Z level=info msg="Executing migration" id="create team table" 2024-09-17T09:25:40.650 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.577788157Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-17T09:25:40.650 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.578502237Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-17T09:25:40.651 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.579512523Z level=info msg="Executing migration" id="create team member table" 2024-09-17T09:25:40.651 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.580214323Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-17T09:25:40.651 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.58117787Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-17T09:25:40.651 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.581919531Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-17T09:25:40.651 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.582948781Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-17T09:25:40.651 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.583711244Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-17T09:25:40.652 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.584544137Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-17T09:25:40.652 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.585393625Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-17T09:25:40.652 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.58627879Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-17T09:25:40.652 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.587283881Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-17T09:25:40.652 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.587951866Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-17T09:25:40.652 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.59068167Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-17T09:25:40.653 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.593538805Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-17T09:25:40.653 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.596168975Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-17T09:25:40.653 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.596638993Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-17T09:25:40.653 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.599438483Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-17T09:25:40.654 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.602110244Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-17T09:25:40.654 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.604894702Z level=info msg="Executing migration" id="create tag table" 2024-09-17T09:25:40.654 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.605440733Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-17T09:25:40.654 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.608324987Z level=info msg="Executing migration" id="create login attempt table" 2024-09-17T09:25:40.654 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.61099585Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-17T09:25:40.654 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.613776106Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-17T09:25:40.654 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.614224703Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-17T09:25:40.655 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.61702404Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-17T09:25:40.655 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.619732475Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-17T09:25:40.655 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.622521431Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-17T09:25:40.655 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.622862155Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-17T09:25:40.655 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.625705807Z level=info msg="Executing migration" id="create user auth table" 2024-09-17T09:25:40.656 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.628397587Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-17T09:25:40.876 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-mon[30659]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:40.877 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:40.877 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:40.877 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:40.877 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:40.878 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.630944436Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-17T09:25:40.878 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.631277866Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-17T09:25:40.878 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.633961046Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-17T09:25:40.878 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.636648959Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-17T09:25:40.878 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.639245747Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-17T09:25:40.878 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.639985223Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-17T09:25:40.878 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.64268676Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-17T09:25:40.878 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.645361077Z level=info msg="Executing migration" id="create server_lock table" 2024-09-17T09:25:40.879 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.648083402Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-17T09:25:40.879 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.648537498Z level=info msg="Executing migration" id="create user auth token table" 2024-09-17T09:25:40.879 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.65128172Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-17T09:25:40.879 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.654048244Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-17T09:25:40.879 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.656671732Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-17T09:25:40.879 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.657166631Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-17T09:25:40.879 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.659893481Z level=info msg="Executing migration" id="create cache_data table" 2024-09-17T09:25:40.879 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.662618608Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-17T09:25:40.879 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.665104528Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-17T09:25:40.879 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.66553929Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-17T09:25:40.879 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.666302423Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-17T09:25:40.880 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.667263313Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-17T09:25:40.880 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.667964192Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-17T09:25:40.880 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.668399362Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-17T09:25:40.880 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.669188723Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-17T09:25:40.880 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.669959902Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-17T09:25:40.880 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.670826823Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-17T09:25:40.880 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.671618719Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-17T09:25:40.880 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.672386409Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-17T09:25:40.880 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.67352951Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-17T09:25:40.880 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.673986531Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-17T09:25:40.880 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.674794265Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-17T09:25:40.881 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.675588291Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-17T09:25:40.881 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.676573788Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-17T09:25:40.881 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.677274882Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-17T09:25:40.881 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.677741845Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-17T09:25:40.881 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.678736305Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-17T09:25:40.881 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.679487426Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-17T09:25:40.881 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.680325883Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-17T09:25:40.881 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.680809883Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-17T09:25:40.881 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.681620723Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-17T09:25:40.881 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.682584579Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-17T09:25:40.881 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.683524218Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-17T09:25:40.881 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.684004597Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-17T09:25:40.881 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.68484703Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-17T09:25:40.882 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.688311128Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-17T09:25:40.882 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.691678758Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-17T09:25:40.882 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.69220629Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-17T09:25:40.882 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.693064694Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-17T09:25:40.882 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.69391543Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-17T09:25:40.882 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.694733584Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-17T09:25:40.882 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.695204474Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-17T09:25:40.882 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.696159602Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-17T09:25:40.882 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.696894793Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-17T09:25:40.882 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.699692547Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-17T09:25:40.882 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.700542386Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-17T09:25:40.882 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.703354576Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-17T09:25:40.883 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.706086652Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-17T09:25:40.883 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.709003004Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-17T09:25:40.883 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.709600709Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-17T09:25:40.883 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.712418137Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-17T09:25:40.883 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.715225534Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-17T09:25:40.883 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.717894928Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-17T09:25:40.883 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.718704892Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-17T09:25:40.883 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.719543685Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-17T09:25:40.883 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.720508805Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-17T09:25:40.883 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.721120798Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-17T09:25:40.883 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.721592123Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-17T09:25:40.883 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.722054613Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-17T09:25:40.884 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.722378666Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-17T09:25:40.884 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.723268958Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-17T09:25:40.884 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.724160849Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-17T09:25:40.884 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.725022426Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-17T09:25:40.884 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.725909661Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-17T09:25:40.884 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.726771732Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-17T09:25:40.884 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.727676133Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-17T09:25:40.884 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.728112366Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-17T09:25:40.884 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.728989315Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-17T09:25:40.884 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.729864179Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-17T09:25:40.884 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.730737323Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-17T09:25:40.884 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.731215201Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-17T09:25:40.884 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.734111392Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-17T09:25:40.885 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.736864688Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-17T09:25:40.885 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.739641654Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-17T09:25:40.885 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.740508637Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-17T09:25:40.885 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.743452103Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-17T09:25:40.885 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.746324159Z level=info msg="Executing migration" id="create alert_image table" 2024-09-17T09:25:40.885 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.749028936Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-17T09:25:40.885 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.749533599Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-17T09:25:40.885 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.752442548Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-17T09:25:40.885 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.752906875Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-17T09:25:40.885 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.755223706Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-17T09:25:40.886 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.755319565Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-17T09:25:40.886 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.758009323Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-17T09:25:40.887 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.758398501Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-17T09:25:40.887 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.761131925Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-17T09:25:40.887 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.761329631Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-17T09:25:40.887 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.761431835Z level=info msg="alerts found to migrate" alerts=0 2024-09-17T09:25:40.887 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.761827585Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-17T09:25:40.887 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.763868907Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-17T09:25:40.887 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.766590001Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-17T09:25:40.887 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.76703929Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-17T09:25:40.887 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.769761117Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-17T09:25:40.887 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.772446693Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-17T09:25:40.887 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.775175972Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-17T09:25:40.888 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.777820019Z level=info msg="Executing migration" id="create data_keys table" 2024-09-17T09:25:40.888 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.780541064Z level=info msg="Executing migration" id="create secrets table" 2024-09-17T09:25:40.888 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.781017866Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-17T09:25:40.888 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.78528494Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-17T09:25:40.888 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.787915853Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-17T09:25:40.888 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.790685316Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-17T09:25:40.888 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.794897787Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-17T09:25:40.888 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.799060282Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-17T09:25:40.889 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.801602881Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-17T09:25:40.889 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.802090783Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-17T09:25:40.889 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.802441769Z level=info msg="Executing migration" id="create permission table" 2024-09-17T09:25:40.889 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.802849393Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-17T09:25:40.889 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.80328828Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-17T09:25:40.889 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.803730181Z level=info msg="Executing migration" id="create role table" 2024-09-17T09:25:40.889 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.804146663Z level=info msg="Executing migration" id="add column display_name" 2024-09-17T09:25:40.889 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.805103324Z level=info msg="Executing migration" id="add column group_name" 2024-09-17T09:25:40.889 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.806079808Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-17T09:25:40.889 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.806508006Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-17T09:25:40.889 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.806941288Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-17T09:25:40.889 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.80736283Z level=info msg="Executing migration" id="create team role table" 2024-09-17T09:25:40.889 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.80773464Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-17T09:25:40.890 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.808168914Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-17T09:25:40.890 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.808601945Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-17T09:25:40.890 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.809024518Z level=info msg="Executing migration" id="create user role table" 2024-09-17T09:25:40.890 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.80939793Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-17T09:25:40.890 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.809818289Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-17T09:25:40.890 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.810240771Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-17T09:25:40.890 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.810653147Z level=info msg="Executing migration" id="create builtin role table" 2024-09-17T09:25:40.890 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.811069973Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-17T09:25:40.890 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.811490199Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-17T09:25:40.890 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.811957016Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-17T09:25:40.890 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.812906099Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-17T09:25:40.890 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.813726483Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-17T09:25:40.890 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.814186865Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-17T09:25:40.890 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.814570973Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-17T09:25:40.891 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.8149929Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-17T09:25:40.891 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.815360152Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-17T09:25:40.891 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.815781312Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-17T09:25:40.891 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.816748012Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-17T09:25:40.891 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.817150596Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-17T09:25:40.891 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.817571316Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-17T09:25:40.891 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.817828599Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-17T09:25:40.891 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.818218561Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-17T09:25:40.891 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.818641156Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-17T09:25:40.891 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.819131643Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-17T09:25:40.891 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.819658506Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-17T09:25:40.891 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.820000568Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-17T09:25:40.892 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.820369508Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-17T09:25:40.892 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.820783285Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-17T09:25:40.892 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.821729704Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-17T09:25:40.892 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.822041555Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-17T09:25:40.892 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.822429397Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-17T09:25:40.892 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.822845584Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-17T09:25:40.892 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.823257673Z level=info msg="Executing migration" id="add correlation config column" 2024-09-17T09:25:40.892 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.824253388Z level=info msg="Executing migration" id="create entity_events table" 2024-09-17T09:25:40.892 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.824661158Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-17T09:25:40.892 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.825070019Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-17T09:25:40.892 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.82515035Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-17T09:25:40.892 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.825414418Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-17T09:25:40.893 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.825488248Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-17T09:25:40.893 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.827348363Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-17T09:25:40.893 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.827876875Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-17T09:25:40.893 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.83007771Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-17T09:25:40.893 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.830900134Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-17T09:25:40.893 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.83135134Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-17T09:25:40.893 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.832257302Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-17T09:25:40.893 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.832773432Z level=info msg="Executing migration" id="Drop public config table" 2024-09-17T09:25:40.893 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.833187333Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-17T09:25:40.893 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.834003587Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-17T09:25:40.893 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.834477168Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-17T09:25:40.893 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.835449995Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-17T09:25:40.893 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.83617933Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-17T09:25:40.894 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.839147158Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-17T09:25:40.894 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.840162662Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-17T09:25:40.894 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.841173575Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-17T09:25:40.894 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.842020831Z level=info msg="Executing migration" id="add share column" 2024-09-17T09:25:40.894 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.843017605Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-17T09:25:40.894 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.843828914Z level=info msg="Executing migration" id="create file table" 2024-09-17T09:25:40.894 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.844253725Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-17T09:25:40.894 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.845291103Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-17T09:25:40.894 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.8459543Z level=info msg="Executing migration" id="create file_meta table" 2024-09-17T09:25:40.894 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.846334916Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-17T09:25:40.894 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.846758666Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-17T09:25:40.894 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.847733718Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-17T09:25:40.894 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.84846867Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-17T09:25:40.894 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.849284478Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-17T09:25:40.895 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.849698651Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-17T09:25:40.895 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.850686115Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-17T09:25:40.895 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.851687823Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-17T09:25:40.895 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.852489557Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-17T09:25:40.895 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.853277146Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-17T09:25:40.895 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.853585689Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-17T09:25:40.895 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.853956011Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-17T09:25:40.895 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.854957048Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-17T09:25:40.895 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.85774991Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-17T09:25:40.895 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.860518167Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-17T09:25:40.895 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.870305693Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-17T09:25:40.895 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.873099727Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-17T09:25:40.895 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.875982874Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-17T09:25:40.895 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.879577622Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-17T09:25:40.896 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.879925088Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-17T09:25:41.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:40 smithi003 ceph-mon[25908]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:41.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:40 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:41.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:40 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:41.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:40 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:41.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:40 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:41.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:40 smithi183 ceph-mon[30372]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:41.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:40 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:41.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:40 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:41.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:40 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:41.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:40 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:41.252 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.882738533Z level=info msg="Executing migration" id="create folder table" 2024-09-17T09:25:41.253 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.885473925Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-17T09:25:41.253 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.888288189Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-17T09:25:41.253 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.891050874Z level=info msg="Executing migration" id="Update folder title length" 2024-09-17T09:25:41.253 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.893845963Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-17T09:25:41.253 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=migrator t=2024-09-17T09:25:40.896668168Z level=info msg="migrations completed" performed=485 skipped=0 duration=609.699228ms 2024-09-17T09:25:41.253 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=sqlstore t=2024-09-17T09:25:40.897101311Z level=info msg="Created default organization" 2024-09-17T09:25:41.253 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=plugin.loader t=2024-09-17T09:25:40.955079164Z level=info msg="Plugin registered" pluginID=input 2024-09-17T09:25:41.253 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=plugin.loader t=2024-09-17T09:25:40.98197141Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-17T09:25:41.253 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=plugin.loader t=2024-09-17T09:25:40.981991118Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-17T09:25:41.254 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=secrets t=2024-09-17T09:25:40.982025307Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-17T09:25:41.254 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=query_data t=2024-09-17T09:25:40.983672641Z level=info msg="Query Service initialization" 2024-09-17T09:25:41.254 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=live.push_http t=2024-09-17T09:25:40.986387712Z level=info msg="Live Push Gateway initialization" 2024-09-17T09:25:41.387 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:41 smithi003 systemd[1]: Starting Ceph node-exporter.a for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:25:42.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:41 smithi077 ceph-mon[30659]: Deploying daemon node-exporter.a on smithi003 2024-09-17T09:25:42.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:41 smithi077 ceph-mon[30659]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:42.002 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:41 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:42.002 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:41 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=infra.usagestats.collector t=2024-09-17T09:25:41.698099364Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-17T09:25:42.002 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:41 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=provisioning.datasources t=2024-09-17T09:25:41.753347786Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-17T09:25:42.002 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:41 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=provisioning.datasources t=2024-09-17T09:25:41.785136867Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-17T09:25:42.003 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:41 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=provisioning.alerting t=2024-09-17T09:25:41.806340716Z level=info msg="starting to provision alerting" 2024-09-17T09:25:42.003 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:41 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=provisioning.alerting t=2024-09-17T09:25:41.806359418Z level=info msg="finished to provision alerting" 2024-09-17T09:25:42.003 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:41 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=grafanaStorageLogger t=2024-09-17T09:25:41.806558155Z level=info msg="storage starting" 2024-09-17T09:25:42.003 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:41 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=ngalert.state.manager t=2024-09-17T09:25:41.806586907Z level=info msg="Warming state cache for startup" 2024-09-17T09:25:42.003 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:41 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=http.server t=2024-09-17T09:25:41.810361307Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-17T09:25:42.003 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:41 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=ngalert.state.manager t=2024-09-17T09:25:41.816777876Z level=info msg="State cache has been initialized" states=0 duration=10.176508ms 2024-09-17T09:25:42.003 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:41 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=ticker t=2024-09-17T09:25:41.816835435Z level=info msg=starting first_tick=2024-09-17T09:25:50Z 2024-09-17T09:25:42.003 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:41 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=ngalert.multiorg.alertmanager t=2024-09-17T09:25:41.816822054Z level=info msg="starting MultiOrg Alertmanager" 2024-09-17T09:25:42.003 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:41 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=sqlstore.transactions t=2024-09-17T09:25:41.816834325Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-17T09:25:42.003 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:25:41 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a[42779]: logger=sqlstore.transactions t=2024-09-17T09:25:41.864059279Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-17T09:25:42.004 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:41 smithi003 ceph-mon[25908]: Deploying daemon node-exporter.a on smithi003 2024-09-17T09:25:42.004 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:41 smithi003 ceph-mon[25908]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:42.004 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:41 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:42.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:41 smithi183 ceph-mon[30372]: Deploying daemon node-exporter.a on smithi003 2024-09-17T09:25:42.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:41 smithi183 ceph-mon[30372]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:42.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:41 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:42.469 INFO:teuthology.orchestra.run.smithi003.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-17T09:25:42.470 INFO:teuthology.orchestra.run.smithi003.stdout:alertmanager ?:9093,9094 0/1 - 58s smithi003=a;count:1 2024-09-17T09:25:42.470 INFO:teuthology.orchestra.run.smithi003.stdout:grafana ?:3000 0/1 - 55s smithi077=a;count:1 2024-09-17T09:25:42.470 INFO:teuthology.orchestra.run.smithi003.stdout:mgr 3/3 34s ago 2m smithi003=y;smithi077=x;smithi183=z;count:3 2024-09-17T09:25:42.470 INFO:teuthology.orchestra.run.smithi003.stdout:mon 3/3 34s ago 2m smithi003:172.21.15.3=a;smithi077:172.21.15.77=b;smithi183:172.21.15.183=c;count:3 2024-09-17T09:25:42.470 INFO:teuthology.orchestra.run.smithi003.stdout:node-exporter ?:9100 0/3 - 60s smithi003=a;smithi077=b;smithi183=c;count:3 2024-09-17T09:25:42.470 INFO:teuthology.orchestra.run.smithi003.stdout:osd 3 34s ago - 2024-09-17T09:25:42.470 INFO:teuthology.orchestra.run.smithi003.stdout:prometheus ?:9095 1/1 33s ago 63s smithi077=a;count:1 2024-09-17T09:25:42.470 INFO:teuthology.orchestra.run.smithi003.stdout:rgw.foo.a ?:80 1/1 28s ago 62s smithi003=foo.a;count:1 2024-09-17T09:25:42.758 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:42 smithi003 bash[57229]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-17T09:25:43.139 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:42 smithi003 bash[57229]: Getting image source signatures 2024-09-17T09:25:43.139 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:42 smithi003 bash[57229]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-17T09:25:43.139 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:42 smithi003 bash[57229]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-17T09:25:43.139 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:42 smithi003 bash[57229]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-17T09:25:43.164 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-17T09:25:43.378 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:25:44.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:43 smithi183 ceph-mon[30372]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:44.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:43 smithi183 ceph-mon[30372]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:44.246 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:43 smithi003 ceph-mon[25908]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:44.246 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:43 smithi003 ceph-mon[25908]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:44.247 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:43 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: ::ffff:172.21.15.77 - - [17/Sep/2024:09:25:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T09:25:44.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:43 smithi077 ceph-mon[30659]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:44.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:43 smithi077 ceph-mon[30659]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:44.508 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:44 smithi003 bash[57229]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-17T09:25:45.008 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:44 smithi003 bash[57229]: Writing manifest to image destination 2024-09-17T09:25:45.508 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:45 smithi003 podman[57229]: 2024-09-17 09:25:45.150330401 +0000 UTC m=+2.830224107 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-17T09:25:45.915 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:45 smithi003 ceph-mon[25908]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:45.915 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:45 smithi003 podman[57229]: 2024-09-17 09:25:45.778521698 +0000 UTC m=+3.458415403 container create 230fc615521d5791bd7684b11eb5da4869441587d73bbb666efb43fa1310a7fa (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T09:25:46.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:45 smithi183 ceph-mon[30372]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:46.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:45 smithi077 ceph-mon[30659]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:46.381 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 podman[57229]: 2024-09-17 09:25:46.056840769 +0000 UTC m=+3.736734492 container init 230fc615521d5791bd7684b11eb5da4869441587d73bbb666efb43fa1310a7fa (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T09:25:46.382 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 podman[57229]: 2024-09-17 09:25:46.062343781 +0000 UTC m=+3.742237487 container start 230fc615521d5791bd7684b11eb5da4869441587d73bbb666efb43fa1310a7fa (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T09:25:46.382 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.066Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-17T09:25:46.382 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.067Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-17T09:25:46.382 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-17T09:25:46.382 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-17T09:25:46.382 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-17T09:25:46.382 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-17T09:25:46.382 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-17T09:25:46.382 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=arp 2024-09-17T09:25:46.382 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-17T09:25:46.382 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-17T09:25:46.383 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-17T09:25:46.383 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-17T09:25:46.383 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-17T09:25:46.383 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-17T09:25:46.383 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-17T09:25:46.383 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-17T09:25:46.383 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=edac 2024-09-17T09:25:46.384 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-17T09:25:46.384 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-17T09:25:46.384 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-17T09:25:46.384 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-17T09:25:46.384 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-17T09:25:46.384 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-17T09:25:46.384 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-17T09:25:46.384 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-17T09:25:46.384 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-17T09:25:46.384 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-17T09:25:46.384 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-17T09:25:46.384 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-17T09:25:46.385 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-17T09:25:46.385 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-17T09:25:46.385 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-17T09:25:46.385 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-17T09:25:46.385 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=os 2024-09-17T09:25:46.385 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-17T09:25:46.385 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-17T09:25:46.385 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-17T09:25:46.385 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-17T09:25:46.385 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-17T09:25:46.385 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-17T09:25:46.385 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-17T09:25:46.385 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=stat 2024-09-17T09:25:46.386 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-17T09:25:46.386 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-17T09:25:46.386 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-17T09:25:46.386 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=time 2024-09-17T09:25:46.386 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-17T09:25:46.386 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=uname 2024-09-17T09:25:46.386 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-17T09:25:46.386 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-17T09:25:46.386 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.068Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-17T09:25:46.386 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.069Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-17T09:25:46.386 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a[57465]: ts=2024-09-17T09:25:46.069Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-17T09:25:46.386 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 bash[57229]: 230fc615521d5791bd7684b11eb5da4869441587d73bbb666efb43fa1310a7fa 2024-09-17T09:25:46.386 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:25:46 smithi003 systemd[1]: Started Ceph node-exporter.a for 08f126da-74d6-11ef-bceb-c7b262605968. 2024-09-17T09:25:46.792 INFO:teuthology.orchestra.run.smithi003.stdout:HOST ADDR LABELS STATUS 2024-09-17T09:25:46.792 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003 172.21.15.3 2024-09-17T09:25:46.793 INFO:teuthology.orchestra.run.smithi003.stdout:smithi077 172.21.15.77 2024-09-17T09:25:46.793 INFO:teuthology.orchestra.run.smithi003.stdout:smithi183 172.21.15.183 2024-09-17T09:25:46.793 INFO:teuthology.orchestra.run.smithi003.stdout:3 hosts in cluster 2024-09-17T09:25:47.355 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:47 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:47.356 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:47 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:47.356 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:47 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:47.356 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:47 smithi003 ceph-mon[25908]: Deploying daemon node-exporter.b on smithi077 2024-09-17T09:25:47.377 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-17T09:25:47.472 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:47 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:47.472 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:47 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:47.472 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:47 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:47.472 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:47 smithi077 ceph-mon[30659]: Deploying daemon node-exporter.b on smithi077 2024-09-17T09:25:47.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:47 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:47.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:47 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:47.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:47 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:47.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:47 smithi183 ceph-mon[30372]: Deploying daemon node-exporter.b on smithi077 2024-09-17T09:25:47.582 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:25:47.752 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:47 smithi077 bash[43047]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-17T09:25:48.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:48 smithi077 ceph-mon[30659]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:48.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:48 smithi077 ceph-mon[30659]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:48.252 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:47 smithi077 bash[43047]: Getting image source signatures 2024-09-17T09:25:48.252 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:47 smithi077 bash[43047]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-17T09:25:48.252 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:47 smithi077 bash[43047]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-17T09:25:48.252 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:47 smithi077 bash[43047]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-17T09:25:48.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:48 smithi003 ceph-mon[25908]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:48.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:48 smithi003 ceph-mon[25908]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:48.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:48 smithi183 ceph-mon[30372]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:48.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:48 smithi183 ceph-mon[30372]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:49.597 INFO:teuthology.orchestra.run.smithi003.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-17T09:25:49.597 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330009Z400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-17T09:25:49.597 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003 /dev/nvme1n1 ssd Linux_fadcd9eb6a1368541422 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-17T09:25:49.597 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003 /dev/nvme2n1 ssd Linux_17f079916edabdad973e 89.4G Yes 2m ago 2024-09-17T09:25:49.597 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003 /dev/nvme3n1 ssd Linux_778b7aa51d6954a1a7b0 89.4G Yes 2m ago 2024-09-17T09:25:49.597 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003 /dev/nvme4n1 ssd Linux_0164f077ff90eda624f2 89.4G Yes 2m ago 2024-09-17T09:25:49.598 INFO:teuthology.orchestra.run.smithi003.stdout:smithi077 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400ZJ400BGN 372G No 105s ago Has a FileSystem, LVM detected 2024-09-17T09:25:49.598 INFO:teuthology.orchestra.run.smithi003.stdout:smithi077 /dev/nvme1n1 ssd Linux_0438b117b1dab8fd7af6 89.4G No 105s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-17T09:25:49.598 INFO:teuthology.orchestra.run.smithi003.stdout:smithi077 /dev/nvme2n1 ssd Linux_cb392e9b894c782dc8a0 89.4G Yes 105s ago 2024-09-17T09:25:49.598 INFO:teuthology.orchestra.run.smithi003.stdout:smithi077 /dev/nvme3n1 ssd Linux_5ccc6d439af993f9767d 89.4G Yes 105s ago 2024-09-17T09:25:49.598 INFO:teuthology.orchestra.run.smithi003.stdout:smithi077 /dev/nvme4n1 ssd Linux_e99ac0a2a44b99ee9daa 89.4G Yes 105s ago 2024-09-17T09:25:49.598 INFO:teuthology.orchestra.run.smithi003.stdout:smithi183 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500QJ400BGN 372G No 83s ago Has a FileSystem, LVM detected 2024-09-17T09:25:49.598 INFO:teuthology.orchestra.run.smithi003.stdout:smithi183 /dev/nvme1n1 ssd Linux_8801d8bb2bb9c79910f3 89.4G No 83s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-17T09:25:49.598 INFO:teuthology.orchestra.run.smithi003.stdout:smithi183 /dev/nvme2n1 ssd Linux_12079317b919e8de678b 89.4G Yes 83s ago 2024-09-17T09:25:49.598 INFO:teuthology.orchestra.run.smithi003.stdout:smithi183 /dev/nvme3n1 ssd Linux_b1019c4afc618e8735fc 89.4G Yes 83s ago 2024-09-17T09:25:49.598 INFO:teuthology.orchestra.run.smithi003.stdout:smithi183 /dev/nvme4n1 ssd Linux_c4b0d2b9775d9062e496 89.4G Yes 83s ago 2024-09-17T09:25:49.752 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:49 smithi077 bash[43047]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-17T09:25:49.752 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:49 smithi077 bash[43047]: Writing manifest to image destination 2024-09-17T09:25:50.069 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:49 smithi077 podman[43047]: 2024-09-17 09:25:49.735492601 +0000 UTC m=+2.170238922 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-17T09:25:50.070 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:49 smithi077 podman[43047]: 2024-09-17 09:25:49.850142529 +0000 UTC m=+2.284888887 container create eecfab806218b74976a665bf01874b5b128cf04556b46fb8750de880532ba275 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T09:25:50.070 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 podman[43047]: 2024-09-17 09:25:50.068835428 +0000 UTC m=+2.503581754 container init eecfab806218b74976a665bf01874b5b128cf04556b46fb8750de880532ba275 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T09:25:50.222 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls --format yaml' 2024-09-17T09:25:50.429 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:25:50.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-mon[30659]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:50.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:50.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:50.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:50.503 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 podman[43047]: 2024-09-17 09:25:50.07436228 +0000 UTC m=+2.509108597 container start eecfab806218b74976a665bf01874b5b128cf04556b46fb8750de880532ba275 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T09:25:50.503 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.078Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-17T09:25:50.503 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.078Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-17T09:25:50.503 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.079Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-17T09:25:50.503 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.079Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-17T09:25:50.503 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.079Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-17T09:25:50.503 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-17T09:25:50.503 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-17T09:25:50.504 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=arp 2024-09-17T09:25:50.504 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-17T09:25:50.504 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-17T09:25:50.504 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-17T09:25:50.504 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-17T09:25:50.504 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-17T09:25:50.504 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-17T09:25:50.504 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-17T09:25:50.504 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-17T09:25:50.504 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=edac 2024-09-17T09:25:50.504 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-17T09:25:50.504 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-17T09:25:50.504 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-17T09:25:50.505 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-17T09:25:50.505 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-17T09:25:50.505 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-17T09:25:50.505 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-17T09:25:50.505 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-17T09:25:50.505 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-17T09:25:50.505 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-17T09:25:50.505 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-17T09:25:50.505 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-17T09:25:50.505 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-17T09:25:50.505 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-17T09:25:50.505 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-17T09:25:50.505 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-17T09:25:50.505 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=os 2024-09-17T09:25:50.506 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-17T09:25:50.506 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-17T09:25:50.506 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-17T09:25:50.506 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-17T09:25:50.506 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-17T09:25:50.506 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-17T09:25:50.506 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-17T09:25:50.506 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=stat 2024-09-17T09:25:50.506 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-17T09:25:50.507 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-17T09:25:50.507 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-17T09:25:50.507 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=time 2024-09-17T09:25:50.507 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-17T09:25:50.507 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=uname 2024-09-17T09:25:50.507 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-17T09:25:50.507 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-17T09:25:50.507 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-17T09:25:50.507 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-17T09:25:50.507 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b[43120]: ts=2024-09-17T09:25:50.080Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-17T09:25:50.507 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 bash[43047]: eecfab806218b74976a665bf01874b5b128cf04556b46fb8750de880532ba275 2024-09-17T09:25:50.507 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:25:50 smithi077 systemd[1]: Started Ceph node-exporter.b for 08f126da-74d6-11ef-bceb-c7b262605968. 2024-09-17T09:25:50.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:50 smithi003 ceph-mon[25908]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:50.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:50 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:50.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:50 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:50.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:50 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:50.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:50 smithi183 ceph-mon[30372]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:50.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:50 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:50.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:50 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:50.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:50 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:51.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:51 smithi183 ceph-mon[30372]: from='client.14565 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:51.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:51 smithi183 ceph-mon[30372]: Deploying daemon node-exporter.c on smithi183 2024-09-17T09:25:51.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:51 smithi077 ceph-mon[30659]: from='client.14565 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:51.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:51 smithi077 ceph-mon[30659]: Deploying daemon node-exporter.c on smithi183 2024-09-17T09:25:51.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:51 smithi003 ceph-mon[25908]: from='client.14565 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:51.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:51 smithi003 ceph-mon[25908]: Deploying daemon node-exporter.c on smithi183 2024-09-17T09:25:52.064 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:51 smithi183 bash[43126]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-17T09:25:52.256 INFO:teuthology.orchestra.run.smithi003.stdout:service_type: alertmanager 2024-09-17T09:25:52.257 INFO:teuthology.orchestra.run.smithi003.stdout:service_name: alertmanager 2024-09-17T09:25:52.257 INFO:teuthology.orchestra.run.smithi003.stdout:placement: 2024-09-17T09:25:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: count: 1 2024-09-17T09:25:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: hosts: 2024-09-17T09:25:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: - smithi003=a 2024-09-17T09:25:52.257 INFO:teuthology.orchestra.run.smithi003.stdout:status: 2024-09-17T09:25:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: created: '2024-09-17T09:24:44.248565Z' 2024-09-17T09:25:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: ports: 2024-09-17T09:25:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: - 9093 2024-09-17T09:25:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: - 9094 2024-09-17T09:25:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: running: 0 2024-09-17T09:25:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: size: 1 2024-09-17T09:25:52.258 INFO:teuthology.orchestra.run.smithi003.stdout:events: 2024-09-17T09:25:52.258 INFO:teuthology.orchestra.run.smithi003.stdout:- 2024-09-17T09:25:29.681071Z service:alertmanager [INFO] "service was created" 2024-09-17T09:25:52.258 INFO:teuthology.orchestra.run.smithi003.stdout:--- 2024-09-17T09:25:52.258 INFO:teuthology.orchestra.run.smithi003.stdout:service_type: grafana 2024-09-17T09:25:52.258 INFO:teuthology.orchestra.run.smithi003.stdout:service_name: grafana 2024-09-17T09:25:52.258 INFO:teuthology.orchestra.run.smithi003.stdout:placement: 2024-09-17T09:25:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: count: 1 2024-09-17T09:25:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: hosts: 2024-09-17T09:25:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: - smithi077=a 2024-09-17T09:25:52.258 INFO:teuthology.orchestra.run.smithi003.stdout:spec: 2024-09-17T09:25:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: anonymous_access: true 2024-09-17T09:25:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: protocol: https 2024-09-17T09:25:52.259 INFO:teuthology.orchestra.run.smithi003.stdout:status: 2024-09-17T09:25:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: created: '2024-09-17T09:24:46.643762Z' 2024-09-17T09:25:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: ports: 2024-09-17T09:25:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: - 3000 2024-09-17T09:25:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: running: 0 2024-09-17T09:25:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: size: 1 2024-09-17T09:25:52.259 INFO:teuthology.orchestra.run.smithi003.stdout:events: 2024-09-17T09:25:52.259 INFO:teuthology.orchestra.run.smithi003.stdout:- 2024-09-17T09:25:40.294219Z service:grafana [INFO] "service was created" 2024-09-17T09:25:52.259 INFO:teuthology.orchestra.run.smithi003.stdout:--- 2024-09-17T09:25:52.259 INFO:teuthology.orchestra.run.smithi003.stdout:service_type: mgr 2024-09-17T09:25:52.259 INFO:teuthology.orchestra.run.smithi003.stdout:service_name: mgr 2024-09-17T09:25:52.259 INFO:teuthology.orchestra.run.smithi003.stdout:placement: 2024-09-17T09:25:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: count: 3 2024-09-17T09:25:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: hosts: 2024-09-17T09:25:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: - smithi003=y 2024-09-17T09:25:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: - smithi077=x 2024-09-17T09:25:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: - smithi183=z 2024-09-17T09:25:52.260 INFO:teuthology.orchestra.run.smithi003.stdout:status: 2024-09-17T09:25:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: created: '2024-09-17T09:23:12.078759Z' 2024-09-17T09:25:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: last_refresh: '2024-09-17T09:25:07.876297Z' 2024-09-17T09:25:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: running: 3 2024-09-17T09:25:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: size: 3 2024-09-17T09:25:52.260 INFO:teuthology.orchestra.run.smithi003.stdout:--- 2024-09-17T09:25:52.260 INFO:teuthology.orchestra.run.smithi003.stdout:service_type: mon 2024-09-17T09:25:52.260 INFO:teuthology.orchestra.run.smithi003.stdout:service_name: mon 2024-09-17T09:25:52.260 INFO:teuthology.orchestra.run.smithi003.stdout:placement: 2024-09-17T09:25:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: count: 3 2024-09-17T09:25:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: hosts: 2024-09-17T09:25:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: - smithi003:172.21.15.3=a 2024-09-17T09:25:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: - smithi077:172.21.15.77=b 2024-09-17T09:25:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: - smithi183:172.21.15.183=c 2024-09-17T09:25:52.261 INFO:teuthology.orchestra.run.smithi003.stdout:status: 2024-09-17T09:25:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: created: '2024-09-17T09:22:46.883312Z' 2024-09-17T09:25:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: last_refresh: '2024-09-17T09:25:07.876209Z' 2024-09-17T09:25:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: running: 3 2024-09-17T09:25:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: size: 3 2024-09-17T09:25:52.261 INFO:teuthology.orchestra.run.smithi003.stdout:--- 2024-09-17T09:25:52.261 INFO:teuthology.orchestra.run.smithi003.stdout:service_type: node-exporter 2024-09-17T09:25:52.261 INFO:teuthology.orchestra.run.smithi003.stdout:service_name: node-exporter 2024-09-17T09:25:52.261 INFO:teuthology.orchestra.run.smithi003.stdout:placement: 2024-09-17T09:25:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: count: 3 2024-09-17T09:25:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: hosts: 2024-09-17T09:25:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: - smithi003=a 2024-09-17T09:25:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: - smithi077=b 2024-09-17T09:25:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: - smithi183=c 2024-09-17T09:25:52.262 INFO:teuthology.orchestra.run.smithi003.stdout:status: 2024-09-17T09:25:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: created: '2024-09-17T09:24:41.624396Z' 2024-09-17T09:25:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: ports: 2024-09-17T09:25:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: - 9100 2024-09-17T09:25:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: running: 0 2024-09-17T09:25:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: size: 3 2024-09-17T09:25:52.262 INFO:teuthology.orchestra.run.smithi003.stdout:events: 2024-09-17T09:25:52.263 INFO:teuthology.orchestra.run.smithi003.stdout:- 2024-09-17T09:25:50.224856Z service:node-exporter [INFO] "service was created" 2024-09-17T09:25:52.263 INFO:teuthology.orchestra.run.smithi003.stdout:--- 2024-09-17T09:25:52.263 INFO:teuthology.orchestra.run.smithi003.stdout:service_type: osd 2024-09-17T09:25:52.263 INFO:teuthology.orchestra.run.smithi003.stdout:service_name: osd 2024-09-17T09:25:52.263 INFO:teuthology.orchestra.run.smithi003.stdout:unmanaged: true 2024-09-17T09:25:52.263 INFO:teuthology.orchestra.run.smithi003.stdout:spec: 2024-09-17T09:25:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: filter_logic: AND 2024-09-17T09:25:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: objectstore: bluestore 2024-09-17T09:25:52.263 INFO:teuthology.orchestra.run.smithi003.stdout:status: 2024-09-17T09:25:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: container_image_id: 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 2024-09-17T09:25:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: container_image_name: quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T09:25:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: last_refresh: '2024-09-17T09:25:07.876362Z' 2024-09-17T09:25:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: running: 3 2024-09-17T09:25:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: size: 3 2024-09-17T09:25:52.264 INFO:teuthology.orchestra.run.smithi003.stdout:--- 2024-09-17T09:25:52.264 INFO:teuthology.orchestra.run.smithi003.stdout:service_type: prometheus 2024-09-17T09:25:52.264 INFO:teuthology.orchestra.run.smithi003.stdout:service_name: prometheus 2024-09-17T09:25:52.264 INFO:teuthology.orchestra.run.smithi003.stdout:placement: 2024-09-17T09:25:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: count: 1 2024-09-17T09:25:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: hosts: 2024-09-17T09:25:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: - smithi077=a 2024-09-17T09:25:52.265 INFO:teuthology.orchestra.run.smithi003.stdout:status: 2024-09-17T09:25:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: created: '2024-09-17T09:24:39.268279Z' 2024-09-17T09:25:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: last_refresh: '2024-09-17T09:25:08.640602Z' 2024-09-17T09:25:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: ports: 2024-09-17T09:25:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: - 9095 2024-09-17T09:25:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: running: 1 2024-09-17T09:25:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: size: 1 2024-09-17T09:25:52.265 INFO:teuthology.orchestra.run.smithi003.stdout:--- 2024-09-17T09:25:52.265 INFO:teuthology.orchestra.run.smithi003.stdout:service_type: rgw 2024-09-17T09:25:52.266 INFO:teuthology.orchestra.run.smithi003.stdout:service_id: foo.a 2024-09-17T09:25:52.266 INFO:teuthology.orchestra.run.smithi003.stdout:service_name: rgw.foo.a 2024-09-17T09:25:52.266 INFO:teuthology.orchestra.run.smithi003.stdout:placement: 2024-09-17T09:25:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: count: 1 2024-09-17T09:25:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: hosts: 2024-09-17T09:25:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: - smithi003=foo.a 2024-09-17T09:25:52.266 INFO:teuthology.orchestra.run.smithi003.stdout:status: 2024-09-17T09:25:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: created: '2024-09-17T09:24:40.015600Z' 2024-09-17T09:25:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: last_refresh: '2024-09-17T09:25:14.426653Z' 2024-09-17T09:25:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: ports: 2024-09-17T09:25:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: - 80 2024-09-17T09:25:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: running: 1 2024-09-17T09:25:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: size: 1 2024-09-17T09:25:52.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:52 smithi077 ceph-mon[30659]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:52.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:52 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:25:52.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:52 smithi003 ceph-mon[25908]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:52.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:52 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:25:52.563 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:52 smithi183 ceph-mon[30372]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:52.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:52 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:25:52.913 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 08f126da-74d6-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls | grep '"'"'^osd '"'"'' 2024-09-17T09:25:53.113 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:25:53.314 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 bash[43126]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-17T09:25:53.314 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 bash[43126]: Writing manifest to image destination 2024-09-17T09:25:53.814 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 podman[43126]: 2024-09-17 09:25:53.522164734 +0000 UTC m=+2.419576060 container create 6cf27fa33a856991932a3448cc28a58101dfbe0cdb522c43741c60bdc3afabb2 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c, maintainer=The Prometheus Authors ) 2024-09-17T09:25:53.814 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 podman[43126]: 2024-09-17 09:25:53.432362957 +0000 UTC m=+2.329774250 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-17T09:25:53.814 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 podman[43126]: 2024-09-17 09:25:53.740647824 +0000 UTC m=+2.638059112 container init 6cf27fa33a856991932a3448cc28a58101dfbe0cdb522c43741c60bdc3afabb2 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c, maintainer=The Prometheus Authors ) 2024-09-17T09:25:53.814 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 podman[43126]: 2024-09-17 09:25:53.745484303 +0000 UTC m=+2.642895585 container start 6cf27fa33a856991932a3448cc28a58101dfbe0cdb522c43741c60bdc3afabb2 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c, maintainer=The Prometheus Authors ) 2024-09-17T09:25:53.814 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.750Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-17T09:25:53.815 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.750Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-17T09:25:53.815 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.750Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-17T09:25:53.815 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.750Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-17T09:25:53.815 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-17T09:25:53.815 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-17T09:25:53.815 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-17T09:25:53.815 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=arp 2024-09-17T09:25:53.815 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-17T09:25:53.815 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-17T09:25:53.815 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-17T09:25:53.815 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-17T09:25:53.815 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-17T09:25:53.815 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-17T09:25:53.816 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-17T09:25:53.816 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-17T09:25:53.816 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=edac 2024-09-17T09:25:53.816 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-17T09:25:53.816 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-17T09:25:53.816 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-17T09:25:53.816 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-17T09:25:53.816 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-17T09:25:53.816 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-17T09:25:53.816 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-17T09:25:53.816 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-17T09:25:53.816 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-17T09:25:53.816 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-17T09:25:53.817 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-17T09:25:53.817 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-17T09:25:53.817 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-17T09:25:53.817 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-17T09:25:53.817 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-17T09:25:53.817 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-17T09:25:53.817 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=os 2024-09-17T09:25:53.817 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-17T09:25:53.817 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-17T09:25:53.818 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-17T09:25:53.818 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-17T09:25:53.818 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-17T09:25:53.818 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-17T09:25:53.818 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-17T09:25:53.818 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=stat 2024-09-17T09:25:53.818 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-17T09:25:53.818 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-17T09:25:53.818 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-17T09:25:53.818 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=time 2024-09-17T09:25:53.818 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-17T09:25:53.818 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=uname 2024-09-17T09:25:53.818 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-17T09:25:53.819 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-17T09:25:53.819 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.751Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-17T09:25:53.819 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.752Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-17T09:25:53.819 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c[43201]: ts=2024-09-17T09:25:53.752Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-17T09:25:54.197 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:53 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: ::ffff:172.21.15.77 - - [17/Sep/2024:09:25:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T09:25:54.240 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 bash[43126]: 6cf27fa33a856991932a3448cc28a58101dfbe0cdb522c43741c60bdc3afabb2 2024-09-17T09:25:54.240 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:25:53 smithi183 systemd[1]: Started Ceph node-exporter.c for 08f126da-74d6-11ef-bceb-c7b262605968. 2024-09-17T09:25:54.491 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:54 smithi003 ceph-mon[25908]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-17T09:25:54.491 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:54 smithi003 ceph-mon[25908]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:54.491 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:54 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:54.491 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:54 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:54.491 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:54 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:54.491 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:54 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:54.492 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:54 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:54.492 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:54 smithi003 ceph-mon[25908]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:54.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:54 smithi077 ceph-mon[30659]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-17T09:25:54.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:54 smithi077 ceph-mon[30659]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:54.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:54 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:54.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:54 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:54.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:54 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:54.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:54 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:54.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:54 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:54.502 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:54 smithi077 ceph-mon[30659]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:54.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:54 smithi183 ceph-mon[30372]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-17T09:25:54.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:54 smithi183 ceph-mon[30372]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:54.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:54 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:54.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:54 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:54.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:54 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:54.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:54 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:54.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:54 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' 2024-09-17T09:25:54.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:54 smithi183 ceph-mon[30372]: from='mgr.14451 172.21.15.3:0/3551844689' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:54.893 INFO:teuthology.orchestra.run.smithi003.stdout:osd 3 47s ago - 2024-09-17T09:25:55.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:55 smithi003 ceph-mon[25908]: pgmap v27: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:55.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:55 smithi003 ceph-mon[25908]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:55.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:55 smithi183 ceph-mon[30372]: pgmap v27: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:55.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:25:55 smithi183 ceph-mon[30372]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:55.569 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-17T09:25:55.578 INFO:tasks.cephadm:Teardown begin 2024-09-17T09:25:55.579 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T09:25:55.609 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T09:25:55.636 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:55 smithi077 ceph-mon[30659]: pgmap v27: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:25:55.637 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:55 smithi077 ceph-mon[30659]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:55.641 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T09:25:55.670 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-17T09:25:55.670 DEBUG:teuthology.orchestra.run.smithi003:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-17T09:25:55.688 DEBUG:teuthology.orchestra.run.smithi077:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-17T09:25:55.706 DEBUG:teuthology.orchestra.run.smithi183:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-17T09:25:55.725 INFO:tasks.cephadm:Stopping all daemons... 2024-09-17T09:25:55.726 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-17T09:25:55.726 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-08f126da-74d6-11ef-bceb-c7b262605968@mon.a 2024-09-17T09:25:55.840 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:55 smithi003 systemd[1]: Stopping Ceph mon.a for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:25:56.556 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:56 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-a[25885]: 2024-09-17T09:25:56.276+0000 7f824b79e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:25:56.556 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:56 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-a[25885]: 2024-09-17T09:25:56.276+0000 7f824b79e640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-17T09:25:56.557 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:56 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: [17/Sep/2024:09:25:56] ENGINE Bus STOPPING 2024-09-17T09:25:56.557 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:56 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: [17/Sep/2024:09:25:56] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T09:25:56.557 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:56 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: [17/Sep/2024:09:25:56] ENGINE Bus STOPPED 2024-09-17T09:25:56.557 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:56 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: [17/Sep/2024:09:25:56] ENGINE Bus STARTING 2024-09-17T09:25:56.557 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:56 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: [17/Sep/2024:09:25:56] ENGINE Serving on http://:::9283 2024-09-17T09:25:56.557 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:25:56 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y[26158]: [17/Sep/2024:09:25:56] ENGINE Bus STARTED 2024-09-17T09:25:56.962 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:56 smithi003 podman[58549]: 2024-09-17 09:25:56.55583827 +0000 UTC m=+0.379630757 container died 9b15ead85423fee8d984a43bc8035cfa2a7109fc37a0aabd624aa117ef98a0e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-17T09:25:57.221 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:57 smithi003 podman[58549]: 2024-09-17 09:25:57.035634354 +0000 UTC m=+0.859426833 container cleanup 9b15ead85423fee8d984a43bc8035cfa2a7109fc37a0aabd624aa117ef98a0e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-17T09:25:57.221 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:57 smithi003 bash[58549]: ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-a 2024-09-17T09:25:57.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 17 09:25:57 smithi003 podman[58621]: 2024-09-17 09:25:57.220312127 +0000 UTC m=+0.694907800 container remove 9b15ead85423fee8d984a43bc8035cfa2a7109fc37a0aabd624aa117ef98a0e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-a, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-17T09:25:58.301 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@mon.a.service' 2024-09-17T09:25:58.350 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:25:58.350 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-17T09:25:58.350 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-17T09:25:58.350 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-08f126da-74d6-11ef-bceb-c7b262605968@mon.b 2024-09-17T09:25:58.519 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:58 smithi077 systemd[1]: Stopping Ceph mon.b for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:25:59.649 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:59 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-b[30636]: 2024-09-17T09:25:59.383+0000 7fba5c1ec640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:25:59.649 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:59 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-b[30636]: 2024-09-17T09:25:59.383+0000 7fba5c1ec640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-17T09:25:59.649 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:59 smithi077 podman[43600]: 2024-09-17 09:25:59.567343964 +0000 UTC m=+0.314905205 container died 2f9641ab2c3ab76fbca1c1905ff85e10719940c137d68c97df8dbcab59245177 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, ceph=True) 2024-09-17T09:26:00.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:59 smithi077 podman[43600]: 2024-09-17 09:25:59.933263513 +0000 UTC m=+0.680824797 container cleanup 2f9641ab2c3ab76fbca1c1905ff85e10719940c137d68c97df8dbcab59245177 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b) 2024-09-17T09:26:00.252 INFO:journalctl@ceph.mon.b.smithi077.stdout:Sep 17 09:25:59 smithi077 bash[43600]: ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-b 2024-09-17T09:26:00.617 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@mon.b.service' 2024-09-17T09:26:00.662 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:26:00.662 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-17T09:26:00.662 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-17T09:26:00.662 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-08f126da-74d6-11ef-bceb-c7b262605968@mon.c 2024-09-17T09:26:00.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:26:00 smithi183 systemd[1]: Stopping Ceph mon.c for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:26:01.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:26:00 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-c[30349]: 2024-09-17T09:26:00.954+0000 7f9735044640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:26:01.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:26:00 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-c[30349]: 2024-09-17T09:26:00.954+0000 7f9735044640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-17T09:26:01.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:26:01 smithi183 podman[43645]: 2024-09-17 09:26:01.128028886 +0000 UTC m=+0.275843568 container died d00a181b864177a30934b3e1367ebaa1945b0573d18ee868b53513979a01fcf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-c, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-17T09:26:01.770 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:26:01 smithi183 podman[43645]: 2024-09-17 09:26:01.526974769 +0000 UTC m=+0.674789440 container cleanup d00a181b864177a30934b3e1367ebaa1945b0573d18ee868b53513979a01fcf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-c, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T09:26:01.770 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 17 09:26:01 smithi183 bash[43645]: ceph-08f126da-74d6-11ef-bceb-c7b262605968-mon-c 2024-09-17T09:26:01.901 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@mon.c.service' 2024-09-17T09:26:01.945 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:26:01.945 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-17T09:26:01.946 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-17T09:26:01.946 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-08f126da-74d6-11ef-bceb-c7b262605968@mgr.y 2024-09-17T09:26:02.258 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:26:01 smithi003 systemd[1]: Stopping Ceph mgr.y for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:26:02.619 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:26:02 smithi003 podman[58851]: 2024-09-17 09:26:02.293859545 +0000 UTC m=+0.151696648 container died e24d87b96bc5ef7e85a104982e7819416c3429a915370b40c4b8923e277b8e8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-17T09:26:03.008 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:26:02 smithi003 podman[58851]: 2024-09-17 09:26:02.618744801 +0000 UTC m=+0.476581905 container cleanup e24d87b96bc5ef7e85a104982e7819416c3429a915370b40c4b8923e277b8e8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T09:26:03.008 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:26:02 smithi003 bash[58851]: ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y 2024-09-17T09:26:03.008 INFO:journalctl@ceph.mgr.y.smithi003.stdout:Sep 17 09:26:02 smithi003 podman[58867]: 2024-09-17 09:26:02.828557544 +0000 UTC m=+0.534290088 container remove e24d87b96bc5ef7e85a104982e7819416c3429a915370b40c4b8923e277b8e8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T09:26:03.066 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-17T09:26:03.103 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:26:03.103 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-17T09:26:03.104 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-17T09:26:03.104 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-08f126da-74d6-11ef-bceb-c7b262605968@mgr.x 2024-09-17T09:26:03.416 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:26:03 smithi077 systemd[1]: Stopping Ceph mgr.x for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:26:03.685 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:26:03 smithi077 podman[43883]: 2024-09-17 09:26:03.415329032 +0000 UTC m=+0.122864825 container died 0e9c55eeabe0a6660c0cd461469c4a60a82009731fedad038084c970278673dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux ) 2024-09-17T09:26:04.002 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:26:03 smithi077 podman[43883]: 2024-09-17 09:26:03.767179209 +0000 UTC m=+0.474715000 container cleanup 0e9c55eeabe0a6660c0cd461469c4a60a82009731fedad038084c970278673dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x, RELEASE=reef-19e2a2b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T09:26:04.002 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 17 09:26:03 smithi077 bash[43883]: ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-x 2024-09-17T09:26:04.237 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-17T09:26:04.273 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:26:04.273 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-17T09:26:04.273 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-17T09:26:04.274 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-08f126da-74d6-11ef-bceb-c7b262605968@mgr.z 2024-09-17T09:26:04.564 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:26:04 smithi183 systemd[1]: Stopping Ceph mgr.z for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:26:04.849 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:26:04 smithi183 podman[43774]: 2024-09-17 09:26:04.587418909 +0000 UTC m=+0.121997742 container died 18462c45a5cb44f30e6ef9ddfba886de59914495e8279ba42cfd260cd3999667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T09:26:05.190 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:26:04 smithi183 podman[43774]: 2024-09-17 09:26:04.997181183 +0000 UTC m=+0.531760057 container cleanup 18462c45a5cb44f30e6ef9ddfba886de59914495e8279ba42cfd260cd3999667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2) 2024-09-17T09:26:05.190 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 17 09:26:04 smithi183 bash[43774]: ceph-08f126da-74d6-11ef-bceb-c7b262605968-mgr-z 2024-09-17T09:26:05.341 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-17T09:26:05.375 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:26:05.376 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-17T09:26:05.376 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-17T09:26:05.376 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-08f126da-74d6-11ef-bceb-c7b262605968@osd.0 2024-09-17T09:26:05.675 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:26:05 smithi003 systemd[1]: Stopping Ceph osd.0 for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:26:06.008 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:26:05 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-0[43429]: 2024-09-17T09:26:05.673+0000 7fece11b8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:26:06.008 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:26:05 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-0[43429]: 2024-09-17T09:26:05.673+0000 7fece11b8640 -1 osd.0 31 *** Got signal Terminated *** 2024-09-17T09:26:06.008 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:26:05 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-0[43429]: 2024-09-17T09:26:05.673+0000 7fece11b8640 -1 osd.0 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T09:26:11.505 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:26:11 smithi003 podman[58983]: 2024-09-17 09:26:11.173020261 +0000 UTC m=+5.600580955 container died 65078fd726b41bdb480b232c1160ac9955df3f5eff612bd232ba4ca031df9db7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-0, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-17T09:26:11.506 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:26:11 smithi003 podman[58983]: 2024-09-17 09:26:11.504948913 +0000 UTC m=+5.932509596 container cleanup 65078fd726b41bdb480b232c1160ac9955df3f5eff612bd232ba4ca031df9db7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-17T09:26:11.758 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:26:11 smithi003 bash[58983]: ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-0 2024-09-17T09:26:11.758 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:26:11 smithi003 podman[59003]: 2024-09-17 09:26:11.697847798 +0000 UTC m=+0.523652354 container remove 65078fd726b41bdb480b232c1160ac9955df3f5eff612bd232ba4ca031df9db7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, ceph=True, GIT_CLEAN=True) 2024-09-17T09:26:12.444 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:26:12 smithi003 podman[59101]: 2024-09-17 09:26:12.033763282 +0000 UTC m=+0.016263702 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T09:26:12.444 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:26:12 smithi003 podman[59101]: 2024-09-17 09:26:12.190326908 +0000 UTC m=+0.172827316 container create 0b1bf0828dad7feaa39cd412897ace6eb3bbb508219f598d9bc5fac11fa9c84b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-17T09:26:12.758 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:26:12 smithi003 podman[59101]: 2024-09-17 09:26:12.443413718 +0000 UTC m=+0.425914128 container init 0b1bf0828dad7feaa39cd412897ace6eb3bbb508219f598d9bc5fac11fa9c84b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-17T09:26:12.758 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:26:12 smithi003 podman[59101]: 2024-09-17 09:26:12.448902111 +0000 UTC m=+0.431402519 container start 0b1bf0828dad7feaa39cd412897ace6eb3bbb508219f598d9bc5fac11fa9c84b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-17T09:26:12.759 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:26:12 smithi003 podman[59101]: 2024-09-17 09:26:12.534391952 +0000 UTC m=+0.516892371 container attach 0b1bf0828dad7feaa39cd412897ace6eb3bbb508219f598d9bc5fac11fa9c84b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-17T09:26:12.759 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 17 09:26:12 smithi003 podman[59101]: 2024-09-17 09:26:12.641291606 +0000 UTC m=+0.623792032 container died 0b1bf0828dad7feaa39cd412897ace6eb3bbb508219f598d9bc5fac11fa9c84b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-17T09:26:13.442 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@osd.0.service' 2024-09-17T09:26:13.487 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:26:13.487 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-17T09:26:13.488 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-17T09:26:13.488 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-08f126da-74d6-11ef-bceb-c7b262605968@osd.1 2024-09-17T09:26:13.799 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:26:13 smithi077 systemd[1]: Stopping Ceph osd.1 for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:26:14.252 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:26:13 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-1[35061]: 2024-09-17T09:26:13.797+0000 7fe5ec852640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:26:14.252 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:26:13 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-1[35061]: 2024-09-17T09:26:13.797+0000 7fe5ec852640 -1 osd.1 31 *** Got signal Terminated *** 2024-09-17T09:26:14.252 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:26:13 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-1[35061]: 2024-09-17T09:26:13.797+0000 7fe5ec852640 -1 osd.1 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T09:26:19.530 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:26:19 smithi077 podman[44018]: 2024-09-17 09:26:19.253885319 +0000 UTC m=+5.570530920 container died 982688de797be1e52f921d053557e9e8d3c8b9e0418f652a3c1a91f088a9cf16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-17T09:26:19.972 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:26:19 smithi077 podman[44018]: 2024-09-17 09:26:19.704014741 +0000 UTC m=+6.020660338 container cleanup 982688de797be1e52f921d053557e9e8d3c8b9e0418f652a3c1a91f088a9cf16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS) 2024-09-17T09:26:19.972 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:26:19 smithi077 bash[44018]: ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-1 2024-09-17T09:26:20.252 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:26:19 smithi077 podman[44039]: 2024-09-17 09:26:19.971876215 +0000 UTC m=+0.717976240 container remove 982688de797be1e52f921d053557e9e8d3c8b9e0418f652a3c1a91f088a9cf16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-1, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-17T09:26:20.558 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:26:20 smithi077 podman[44135]: 2024-09-17 09:26:20.181169597 +0000 UTC m=+0.015770418 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T09:26:20.558 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:26:20 smithi077 podman[44135]: 2024-09-17 09:26:20.280936379 +0000 UTC m=+0.115537237 container create 064079cef23195d6c77f00a379b644b4fe40eca3c7be569da14c49bcc502fdad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-17T09:26:20.559 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:26:20 smithi077 podman[44135]: 2024-09-17 09:26:20.474688677 +0000 UTC m=+0.309289494 container init 064079cef23195d6c77f00a379b644b4fe40eca3c7be569da14c49bcc502fdad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-09-17T09:26:20.559 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:26:20 smithi077 podman[44135]: 2024-09-17 09:26:20.479158338 +0000 UTC m=+0.313759151 container start 064079cef23195d6c77f00a379b644b4fe40eca3c7be569da14c49bcc502fdad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T09:26:20.559 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:26:20 smithi077 podman[44135]: 2024-09-17 09:26:20.557866982 +0000 UTC m=+0.392467806 container attach 064079cef23195d6c77f00a379b644b4fe40eca3c7be569da14c49bcc502fdad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-09-17T09:26:20.978 INFO:journalctl@ceph.osd.1.smithi077.stdout:Sep 17 09:26:20 smithi077 podman[44135]: 2024-09-17 09:26:20.662978405 +0000 UTC m=+0.497579219 container died 064079cef23195d6c77f00a379b644b4fe40eca3c7be569da14c49bcc502fdad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-1-deactivate, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-17T09:26:21.278 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@osd.1.service' 2024-09-17T09:26:21.315 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:26:21.315 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-17T09:26:21.315 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-17T09:26:21.316 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-08f126da-74d6-11ef-bceb-c7b262605968@osd.2 2024-09-17T09:26:21.814 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:26:21 smithi183 systemd[1]: Stopping Ceph osd.2 for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:26:21.814 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:26:21 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-2[35479]: 2024-09-17T09:26:21.583+0000 7f4bbf036640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:26:21.814 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:26:21 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-2[35479]: 2024-09-17T09:26:21.583+0000 7f4bbf036640 -1 osd.2 31 *** Got signal Terminated *** 2024-09-17T09:26:21.814 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:26:21 smithi183 ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-2[35479]: 2024-09-17T09:26:21.583+0000 7f4bbf036640 -1 osd.2 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T09:26:27.314 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:26:27 smithi183 podman[43911]: 2024-09-17 09:26:27.032652096 +0000 UTC m=+5.527604308 container died 0c45ea4ba1c683c05c6d1b861de96f103e10f1d212212bf20040f94ca39e3dc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-2, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-17T09:26:27.814 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:26:27 smithi183 podman[43911]: 2024-09-17 09:26:27.406475951 +0000 UTC m=+5.901428217 container cleanup 0c45ea4ba1c683c05c6d1b861de96f103e10f1d212212bf20040f94ca39e3dc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731) 2024-09-17T09:26:27.814 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:26:27 smithi183 bash[43911]: ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-2 2024-09-17T09:26:27.814 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:26:27 smithi183 podman[43944]: 2024-09-17 09:26:27.582285509 +0000 UTC m=+0.550077975 container remove 0c45ea4ba1c683c05c6d1b861de96f103e10f1d212212bf20040f94ca39e3dc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T09:26:28.314 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:26:28 smithi183 podman[44041]: 2024-09-17 09:26:27.91604338 +0000 UTC m=+0.015684539 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T09:26:28.314 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:26:28 smithi183 podman[44041]: 2024-09-17 09:26:28.016340414 +0000 UTC m=+0.115981556 container create 9c0d76d528413e3b84fe86f8112618d061f0fe75d708ed0c29ff8837c0874828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-17T09:26:28.314 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:26:28 smithi183 podman[44041]: 2024-09-17 09:26:28.235323341 +0000 UTC m=+0.334964494 container init 9c0d76d528413e3b84fe86f8112618d061f0fe75d708ed0c29ff8837c0874828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-17T09:26:28.314 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:26:28 smithi183 podman[44041]: 2024-09-17 09:26:28.240455983 +0000 UTC m=+0.340097128 container start 9c0d76d528413e3b84fe86f8112618d061f0fe75d708ed0c29ff8837c0874828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b) 2024-09-17T09:26:28.722 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:26:28 smithi183 podman[44041]: 2024-09-17 09:26:28.318439538 +0000 UTC m=+0.418080711 container attach 9c0d76d528413e3b84fe86f8112618d061f0fe75d708ed0c29ff8837c0874828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T09:26:28.722 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:26:28 smithi183 podman[44041]: 2024-09-17 09:26:28.431012121 +0000 UTC m=+0.530653275 container died 9c0d76d528413e3b84fe86f8112618d061f0fe75d708ed0c29ff8837c0874828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True) 2024-09-17T09:26:28.974 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 17 09:26:28 smithi183 podman[44084]: 2024-09-17 09:26:28.954392071 +0000 UTC m=+0.512831741 container remove 9c0d76d528413e3b84fe86f8112618d061f0fe75d708ed0c29ff8837c0874828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=reef-19e2a2b) 2024-09-17T09:26:28.979 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@osd.2.service' 2024-09-17T09:26:29.017 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:26:29.017 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-17T09:26:29.018 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-17T09:26:29.018 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-08f126da-74d6-11ef-bceb-c7b262605968@rgw.foo.a 2024-09-17T09:26:29.332 INFO:journalctl@ceph.rgw.foo.a.smithi003.stdout:Sep 17 09:26:29 smithi003 systemd[1]: Stopping Ceph rgw.foo.a for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:26:29.758 INFO:journalctl@ceph.rgw.foo.a.smithi003.stdout:Sep 17 09:26:29 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-rgw-foo-a[50333]: 2024-09-17T09:26:29.340+0000 7fbeef30e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:26:29.758 INFO:journalctl@ceph.rgw.foo.a.smithi003.stdout:Sep 17 09:26:29 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-rgw-foo-a[50333]: 2024-09-17T09:26:29.340+0000 7fbef2b79880 -1 shutting down 2024-09-17T09:26:39.632 INFO:journalctl@ceph.rgw.foo.a.smithi003.stdout:Sep 17 09:26:39 smithi003 bash[59230]: time="2024-09-17T09:26:39Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-08f126da-74d6-11ef-bceb-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-17T09:26:39.633 INFO:journalctl@ceph.rgw.foo.a.smithi003.stdout:Sep 17 09:26:39 smithi003 podman[59230]: 2024-09-17 09:26:39.366609699 +0000 UTC m=+10.135670371 container died 13fcd40fc00c69c1ccd942472d27916d35ab9b1d51a41cdb0bef2444cc86b5b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-rgw-foo-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b) 2024-09-17T09:26:40.008 INFO:journalctl@ceph.rgw.foo.a.smithi003.stdout:Sep 17 09:26:39 smithi003 podman[59230]: 2024-09-17 09:26:39.830917045 +0000 UTC m=+10.599977716 container cleanup 13fcd40fc00c69c1ccd942472d27916d35ab9b1d51a41cdb0bef2444cc86b5b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-17T09:26:40.008 INFO:journalctl@ceph.rgw.foo.a.smithi003.stdout:Sep 17 09:26:39 smithi003 bash[59230]: ceph-08f126da-74d6-11ef-bceb-c7b262605968-rgw-foo-a 2024-09-17T09:26:40.274 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@rgw.foo.a.service' 2024-09-17T09:26:40.309 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:26:40.310 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-17T09:26:40.310 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-17T09:26:40.310 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-08f126da-74d6-11ef-bceb-c7b262605968@prometheus.a 2024-09-17T09:26:40.632 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:26:40 smithi077 systemd[1]: Stopping Ceph prometheus.a for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:26:40.897 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:26:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:26:40.630Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-17T09:26:40.897 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:26:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:26:40.631Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-17T09:26:40.897 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:26:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:26:40.631Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-17T09:26:40.897 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:26:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:26:40.631Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-17T09:26:40.897 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:26:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:26:40.631Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-17T09:26:40.897 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:26:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:26:40.631Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-17T09:26:40.898 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:26:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:26:40.631Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-17T09:26:40.898 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:26:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:26:40.631Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-17T09:26:40.898 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:26:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:26:40.631Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-17T09:26:40.898 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:26:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:26:40.631Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-17T09:26:40.898 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:26:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:26:40.631Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-17T09:26:40.898 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:26:40 smithi077 ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a[40947]: ts=2024-09-17T09:26:40.631Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-17T09:26:40.898 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:26:40 smithi077 podman[44262]: 2024-09-17 09:26:40.641305028 +0000 UTC m=+0.141246833 container died baa612d113edfff83e522ed0f700c7cbbb4f1445f05656a1f1ff88155fdf8c13 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T09:26:41.252 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:26:40 smithi077 podman[44262]: 2024-09-17 09:26:40.98686333 +0000 UTC m=+0.486805134 container cleanup baa612d113edfff83e522ed0f700c7cbbb4f1445f05656a1f1ff88155fdf8c13 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T09:26:41.252 INFO:journalctl@ceph.prometheus.a.smithi077.stdout:Sep 17 09:26:40 smithi077 bash[44262]: ceph-08f126da-74d6-11ef-bceb-c7b262605968-prometheus-a 2024-09-17T09:26:41.484 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-08f126da-74d6-11ef-bceb-c7b262605968@prometheus.a.service' 2024-09-17T09:26:41.518 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:26:41.518 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-17T09:26:41.519 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 08f126da-74d6-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-17T09:26:41.678 INFO:teuthology.orchestra.run.smithi003.stdout:Deleting cluster with fsid: 08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:26:42.837 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:26:42 smithi003 systemd[1]: Stopping Ceph alertmanager.a for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:26:43.127 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:26:42 smithi003 ceph-08f126da-74d6-11ef-bceb-c7b262605968-alertmanager-a[55997]: ts=2024-09-17T09:26:42.835Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-17T09:26:43.127 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:26:42 smithi003 podman[59562]: 2024-09-17 09:26:42.856713553 +0000 UTC m=+0.134419188 container died 3379156a6600b24928ac927cb9171e23988727cf28539aa9b61e77d334851c7b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T09:26:43.508 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:26:43 smithi003 podman[59562]: 2024-09-17 09:26:43.308947996 +0000 UTC m=+0.586653691 container cleanup 3379156a6600b24928ac927cb9171e23988727cf28539aa9b61e77d334851c7b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T09:26:43.508 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:26:43 smithi003 bash[59562]: ceph-08f126da-74d6-11ef-bceb-c7b262605968-alertmanager-a 2024-09-17T09:26:43.873 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:26:43 smithi003 podman[59579]: 2024-09-17 09:26:43.601875368 +0000 UTC m=+0.741491502 container remove 3379156a6600b24928ac927cb9171e23988727cf28539aa9b61e77d334851c7b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T09:26:43.873 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:26:43 smithi003 podman[59579]: 2024-09-17 09:26:43.677270613 +0000 UTC m=+0.816886704 volume remove b7d5bff72475a837a333145e91ba3946dc0d21589855af27f0c09dbea35259d1 2024-09-17T09:26:44.258 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:26:44 smithi003 systemd[1]: Stopping Ceph node-exporter.a for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:26:44.259 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:26:43 smithi003 systemd[1]: ceph-08f126da-74d6-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-17T09:26:44.259 INFO:journalctl@ceph.alertmanager.a.smithi003.stdout:Sep 17 09:26:43 smithi003 systemd[1]: Stopped Ceph alertmanager.a for 08f126da-74d6-11ef-bceb-c7b262605968. 2024-09-17T09:26:44.705 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:26:44 smithi003 podman[59693]: 2024-09-17 09:26:44.351090979 +0000 UTC m=+0.127744881 container died 230fc615521d5791bd7684b11eb5da4869441587d73bbb666efb43fa1310a7fa (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T09:26:45.008 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:26:44 smithi003 podman[59693]: 2024-09-17 09:26:44.796695486 +0000 UTC m=+0.573349394 container cleanup 230fc615521d5791bd7684b11eb5da4869441587d73bbb666efb43fa1310a7fa (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T09:26:45.008 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:26:44 smithi003 bash[59693]: ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a 2024-09-17T09:26:45.384 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:26:45 smithi003 podman[59710]: 2024-09-17 09:26:45.064235572 +0000 UTC m=+0.717163498 container remove 230fc615521d5791bd7684b11eb5da4869441587d73bbb666efb43fa1310a7fa (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T09:26:45.385 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:26:45 smithi003 systemd[1]: ceph-08f126da-74d6-11ef-bceb-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-17T09:26:45.385 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:26:45 smithi003 systemd[1]: ceph-08f126da-74d6-11ef-bceb-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-17T09:26:45.385 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:26:45 smithi003 systemd[1]: Stopped Ceph node-exporter.a for 08f126da-74d6-11ef-bceb-c7b262605968. 2024-09-17T09:26:45.385 INFO:journalctl@ceph.node-exporter.a.smithi003.stdout:Sep 17 09:26:45 smithi003 systemd[1]: ceph-08f126da-74d6-11ef-bceb-c7b262605968@node-exporter.a.service: Consumed 1.854s CPU time. 2024-09-17T09:26:45.890 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 08f126da-74d6-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-17T09:26:46.042 INFO:teuthology.orchestra.run.smithi077.stdout:Deleting cluster with fsid: 08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:26:47.002 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:26:46 smithi077 systemd[1]: Stopping Ceph grafana.a for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:26:47.359 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:26:47 smithi077 podman[44586]: 2024-09-17 09:26:47.08746987 +0000 UTC m=+0.124787079 container died 23abc9d6b18fe0e2c1ed1d7b906b8fbfa5dff66d2e5e1fd29a337903fdb7b067 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a, io.openshift.expose-services=, architecture=x86_64, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.component=ubi8-container, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-type=git, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, build-date=2022-04-29T11:48:11.055897, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., version=8.5, io.openshift.tags=base rhel8, name=ubi8, release=239.1651231664, distribution-scope=public, vendor=Red Hat, Inc., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, maintainer=Paul Cuzner , description=Ceph Grafana Container, io.buildah.version=1.32.0, io.k8s.display-name=Red Hat Universal Base Image 8) 2024-09-17T09:26:47.709 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:26:47 smithi077 podman[44586]: 2024-09-17 09:26:47.440855233 +0000 UTC m=+0.478172434 container cleanup 23abc9d6b18fe0e2c1ed1d7b906b8fbfa5dff66d2e5e1fd29a337903fdb7b067 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a, io.openshift.tags=base rhel8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-type=git, io.openshift.expose-services=, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.display-name=Red Hat Universal Base Image 8, vendor=Red Hat, Inc., architecture=x86_64, build-date=2022-04-29T11:48:11.055897, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, description=Ceph Grafana Container, summary=Grafana Container configured for Ceph mgr/dashboard integration, version=8.5, maintainer=Paul Cuzner , io.buildah.version=1.32.0, name=ubi8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, distribution-scope=public, com.redhat.component=ubi8-container) 2024-09-17T09:26:47.709 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:26:47 smithi077 bash[44586]: ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a 2024-09-17T09:26:48.002 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:26:47 smithi077 podman[44604]: 2024-09-17 09:26:47.708566703 +0000 UTC m=+0.623182615 container remove 23abc9d6b18fe0e2c1ed1d7b906b8fbfa5dff66d2e5e1fd29a337903fdb7b067 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-grafana-a, version=8.5, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.buildah.version=1.32.0, distribution-scope=public, vendor=Red Hat, Inc., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.expose-services=, maintainer=Paul Cuzner , description=Ceph Grafana Container, name=ubi8, com.redhat.component=ubi8-container, io.openshift.tags=base rhel8, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, release=239.1651231664, architecture=x86_64, io.k8s.display-name=Red Hat Universal Base Image 8, vcs-type=git, build-date=2022-04-29T11:48:11.055897) 2024-09-17T09:26:48.002 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:26:47 smithi077 systemd[1]: ceph-08f126da-74d6-11ef-bceb-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-17T09:26:48.002 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:26:47 smithi077 systemd[1]: ceph-08f126da-74d6-11ef-bceb-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-17T09:26:48.002 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:26:47 smithi077 systemd[1]: Stopped Ceph grafana.a for 08f126da-74d6-11ef-bceb-c7b262605968. 2024-09-17T09:26:48.002 INFO:journalctl@ceph.grafana.a.smithi077.stdout:Sep 17 09:26:47 smithi077 systemd[1]: ceph-08f126da-74d6-11ef-bceb-c7b262605968@grafana.a.service: Consumed 2.973s CPU time. 2024-09-17T09:26:48.308 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:26:48 smithi077 systemd[1]: Stopping Ceph node-exporter.b for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:26:48.578 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:26:48 smithi077 podman[44714]: 2024-09-17 09:26:48.307578389 +0000 UTC m=+0.120512833 container died eecfab806218b74976a665bf01874b5b128cf04556b46fb8750de880532ba275 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T09:26:48.578 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:26:48 smithi077 podman[44714]: 2024-09-17 09:26:48.577593196 +0000 UTC m=+0.390527629 container cleanup eecfab806218b74976a665bf01874b5b128cf04556b46fb8750de880532ba275 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T09:26:48.968 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:26:48 smithi077 bash[44714]: ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b 2024-09-17T09:26:48.969 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:26:48 smithi077 podman[44732]: 2024-09-17 09:26:48.753844086 +0000 UTC m=+0.450450211 container remove eecfab806218b74976a665bf01874b5b128cf04556b46fb8750de880532ba275 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T09:26:48.969 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:26:48 smithi077 systemd[1]: ceph-08f126da-74d6-11ef-bceb-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-17T09:26:49.252 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:26:48 smithi077 systemd[1]: ceph-08f126da-74d6-11ef-bceb-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-17T09:26:49.252 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:26:48 smithi077 systemd[1]: Stopped Ceph node-exporter.b for 08f126da-74d6-11ef-bceb-c7b262605968. 2024-09-17T09:26:49.252 INFO:journalctl@ceph.node-exporter.b.smithi077.stdout:Sep 17 09:26:48 smithi077 systemd[1]: ceph-08f126da-74d6-11ef-bceb-c7b262605968@node-exporter.b.service: Consumed 1.709s CPU time. 2024-09-17T09:26:49.647 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 08f126da-74d6-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-17T09:26:49.799 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: 08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:26:50.701 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:26:50 smithi183 systemd[1]: Stopping Ceph node-exporter.c for 08f126da-74d6-11ef-bceb-c7b262605968... 2024-09-17T09:26:51.030 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:26:50 smithi183 podman[44316]: 2024-09-17 09:26:50.70051531 +0000 UTC m=+0.117353178 container died 6cf27fa33a856991932a3448cc28a58101dfbe0cdb522c43741c60bdc3afabb2 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c, maintainer=The Prometheus Authors ) 2024-09-17T09:26:51.314 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:26:51 smithi183 podman[44316]: 2024-09-17 09:26:51.028962432 +0000 UTC m=+0.445800293 container cleanup 6cf27fa33a856991932a3448cc28a58101dfbe0cdb522c43741c60bdc3afabb2 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c, maintainer=The Prometheus Authors ) 2024-09-17T09:26:51.314 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:26:51 smithi183 bash[44316]: ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c 2024-09-17T09:26:51.314 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:26:51 smithi183 podman[44333]: 2024-09-17 09:26:51.205218751 +0000 UTC m=+0.508668287 container remove 6cf27fa33a856991932a3448cc28a58101dfbe0cdb522c43741c60bdc3afabb2 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-08f126da-74d6-11ef-bceb-c7b262605968-node-exporter-c, maintainer=The Prometheus Authors ) 2024-09-17T09:26:51.314 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:26:51 smithi183 systemd[1]: ceph-08f126da-74d6-11ef-bceb-c7b262605968@node-exporter.c.service: Main process exited, code=exited, status=143/n/a 2024-09-17T09:26:51.814 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:26:51 smithi183 systemd[1]: ceph-08f126da-74d6-11ef-bceb-c7b262605968@node-exporter.c.service: Failed with result 'exit-code'. 2024-09-17T09:26:51.814 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:26:51 smithi183 systemd[1]: Stopped Ceph node-exporter.c for 08f126da-74d6-11ef-bceb-c7b262605968. 2024-09-17T09:26:51.814 INFO:journalctl@ceph.node-exporter.c.smithi183.stdout:Sep 17 09:26:51 smithi183 systemd[1]: ceph-08f126da-74d6-11ef-bceb-c7b262605968@node-exporter.c.service: Consumed 1.728s CPU time. 2024-09-17T09:26:52.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T09:26:52.114 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T09:26:52.142 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T09:26:52.170 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-17T09:26:52.172 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908293/remote/smithi003/crash 2024-09-17T09:26:52.173 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/crash -- . 2024-09-17T09:26:52.213 INFO:teuthology.orchestra.run.smithi003.stderr:tar: /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-17T09:26:52.213 INFO:teuthology.orchestra.run.smithi003.stderr:tar: Error is not recoverable: exiting now 2024-09-17T09:26:52.215 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908293/remote/smithi077/crash 2024-09-17T09:26:52.216 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/crash -- . 2024-09-17T09:26:52.256 INFO:teuthology.orchestra.run.smithi077.stderr:tar: /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-17T09:26:52.256 INFO:teuthology.orchestra.run.smithi077.stderr:tar: Error is not recoverable: exiting now 2024-09-17T09:26:52.258 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908293/remote/smithi183/crash 2024-09-17T09:26:52.259 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/crash -- . 2024-09-17T09:26:52.297 INFO:teuthology.orchestra.run.smithi183.stderr:tar: /var/lib/ceph/08f126da-74d6-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-17T09:26:52.297 INFO:teuthology.orchestra.run.smithi183.stderr:tar: Error is not recoverable: exiting now 2024-09-17T09:26:52.298 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-17T09:26:52.298 DEBUG:teuthology.orchestra.run.smithi003:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/08f126da-74d6-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v CEPHADM_AGENT_DOWN | egrep -v CEPHADM_FAILED_DAEMON | head -n 1 2024-09-17T09:26:52.327 INFO:teuthology.orchestra.run.smithi003.stderr:grep: /var/log/ceph/08f126da-74d6-11ef-bceb-c7b262605968/ceph.log: No such file or directory 2024-09-17T09:26:52.328 INFO:tasks.cephadm:Compressing logs... 2024-09-17T09:26:52.328 DEBUG:teuthology.orchestra.run.smithi003:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T09:26:52.372 DEBUG:teuthology.orchestra.run.smithi077:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T09:26:52.374 DEBUG:teuthology.orchestra.run.smithi183:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T09:26:52.396 INFO:teuthology.orchestra.run.smithi077.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-17T09:26:52.397 INFO:teuthology.orchestra.run.smithi003.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-17T09:26:52.398 INFO:teuthology.orchestra.run.smithi183.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-17T09:26:53.006 INFO:tasks.cephadm:Archiving logs... 2024-09-17T09:26:53.006 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908293/remote/smithi003/log 2024-09-17T09:26:53.007 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T09:26:53.160 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908293/remote/smithi077/log 2024-09-17T09:26:53.161 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T09:26:53.237 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908293/remote/smithi183/log 2024-09-17T09:26:53.238 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T09:26:53.318 INFO:tasks.cephadm:Removing cluster... 2024-09-17T09:26:53.318 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 08f126da-74d6-11ef-bceb-c7b262605968 --force 2024-09-17T09:26:53.461 INFO:teuthology.orchestra.run.smithi003.stdout:Deleting cluster with fsid: 08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:26:53.591 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 08f126da-74d6-11ef-bceb-c7b262605968 --force 2024-09-17T09:26:53.735 INFO:teuthology.orchestra.run.smithi077.stdout:Deleting cluster with fsid: 08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:26:53.859 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 08f126da-74d6-11ef-bceb-c7b262605968 --force 2024-09-17T09:26:54.007 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: 08f126da-74d6-11ef-bceb-c7b262605968 2024-09-17T09:26:54.134 INFO:tasks.cephadm:Removing cephadm ... 2024-09-17T09:26:54.134 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-17T09:26:54.152 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-17T09:26:54.167 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-17T09:26:54.182 INFO:tasks.cephadm:Teardown complete 2024-09-17T09:26:54.182 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop 2024-09-17T09:26:54.195 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi003:/dev/vg_nvme/lv_1... 2024-09-17T09:26:54.195 DEBUG:teuthology.orchestra.run.smithi003:> sudo nvme disconnect -n lv_1 2024-09-17T09:26:54.346 INFO:teuthology.orchestra.run.smithi003.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-17T09:26:54.348 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi003:/dev/vg_nvme/lv_2... 2024-09-17T09:26:54.348 DEBUG:teuthology.orchestra.run.smithi003:> sudo nvme disconnect -n lv_2 2024-09-17T09:26:54.487 INFO:teuthology.orchestra.run.smithi003.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-17T09:26:54.489 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi003:/dev/vg_nvme/lv_3... 2024-09-17T09:26:54.489 DEBUG:teuthology.orchestra.run.smithi003:> sudo nvme disconnect -n lv_3 2024-09-17T09:26:54.672 INFO:teuthology.orchestra.run.smithi003.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-17T09:26:54.673 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi003:/dev/vg_nvme/lv_4... 2024-09-17T09:26:54.674 DEBUG:teuthology.orchestra.run.smithi003:> sudo nvme disconnect -n lv_4 2024-09-17T09:26:54.839 INFO:teuthology.orchestra.run.smithi003.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-17T09:26:54.840 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-17T09:26:54.840 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/scratch_devs 2024-09-17T09:26:54.867 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi077:/dev/vg_nvme/lv_1... 2024-09-17T09:26:54.867 DEBUG:teuthology.orchestra.run.smithi077:> sudo nvme disconnect -n lv_1 2024-09-17T09:26:55.034 INFO:teuthology.orchestra.run.smithi077.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-17T09:26:55.036 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi077:/dev/vg_nvme/lv_2... 2024-09-17T09:26:55.036 DEBUG:teuthology.orchestra.run.smithi077:> sudo nvme disconnect -n lv_2 2024-09-17T09:26:55.232 INFO:teuthology.orchestra.run.smithi077.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-17T09:26:55.233 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi077:/dev/vg_nvme/lv_3... 2024-09-17T09:26:55.234 DEBUG:teuthology.orchestra.run.smithi077:> sudo nvme disconnect -n lv_3 2024-09-17T09:26:55.410 INFO:teuthology.orchestra.run.smithi077.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-17T09:26:55.412 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi077:/dev/vg_nvme/lv_4... 2024-09-17T09:26:55.412 DEBUG:teuthology.orchestra.run.smithi077:> sudo nvme disconnect -n lv_4 2024-09-17T09:26:55.624 INFO:teuthology.orchestra.run.smithi077.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-17T09:26:55.625 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-17T09:26:55.626 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/scratch_devs 2024-09-17T09:26:55.650 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi183:/dev/vg_nvme/lv_1... 2024-09-17T09:26:55.651 DEBUG:teuthology.orchestra.run.smithi183:> sudo nvme disconnect -n lv_1 2024-09-17T09:26:55.801 INFO:teuthology.orchestra.run.smithi183.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-17T09:26:55.803 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi183:/dev/vg_nvme/lv_2... 2024-09-17T09:26:55.803 DEBUG:teuthology.orchestra.run.smithi183:> sudo nvme disconnect -n lv_2 2024-09-17T09:26:55.945 INFO:teuthology.orchestra.run.smithi183.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-17T09:26:55.947 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi183:/dev/vg_nvme/lv_3... 2024-09-17T09:26:55.947 DEBUG:teuthology.orchestra.run.smithi183:> sudo nvme disconnect -n lv_3 2024-09-17T09:26:56.105 INFO:teuthology.orchestra.run.smithi183.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-17T09:26:56.107 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi183:/dev/vg_nvme/lv_4... 2024-09-17T09:26:56.107 DEBUG:teuthology.orchestra.run.smithi183:> sudo nvme disconnect -n lv_4 2024-09-17T09:26:56.260 INFO:teuthology.orchestra.run.smithi183.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-17T09:26:56.262 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-17T09:26:56.262 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/scratch_devs 2024-09-17T09:26:56.291 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T09:26:56.300 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T09:26:56.300 DEBUG:teuthology.orchestra.run.smithi003:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T09:26:56.304 DEBUG:teuthology.orchestra.run.smithi077:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T09:26:56.307 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T09:26:56.318 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 1: ntpq: command not found 2024-09-17T09:26:56.319 INFO:teuthology.orchestra.run.smithi077.stderr:bash: line 1: ntpq: command not found 2024-09-17T09:26:56.323 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T09:26:56.324 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2024-09-17T09:26:56.324 INFO:teuthology.orchestra.run.smithi003.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 -241us[ -282us] +/- 36ms 2024-09-17T09:26:56.324 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 28 -1017us[-1058us] +/- 57ms 2024-09-17T09:26:56.324 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 +1878us[+1837us] +/- 77ms 2024-09-17T09:26:56.324 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:26:56.325 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T09:26:56.325 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2024-09-17T09:26:56.325 INFO:teuthology.orchestra.run.smithi077.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 -222us[ -229us] +/- 36ms 2024-09-17T09:26:56.325 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 26 -999us[ -999us] +/- 57ms 2024-09-17T09:26:56.325 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 +1915us[+1915us] +/- 77ms 2024-09-17T09:26:56.325 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:26:56.347 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-09-17T09:26:56.352 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T09:26:56.353 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-09-17T09:26:56.353 INFO:teuthology.orchestra.run.smithi183.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 -231us[ -250us] +/- 36ms 2024-09-17T09:26:56.353 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 26 -1008us[-1026us] +/- 57ms 2024-09-17T09:26:56.353 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 +1900us[+1881us] +/- 77ms 2024-09-17T09:26:56.353 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:26:56.354 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T09:26:56.363 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T09:26:56.363 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T09:26:56.410 DEBUG:teuthology.orchestra.run.smithi003:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T09:26:56.486 DEBUG:teuthology.orchestra.run.smithi077:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T09:26:56.550 DEBUG:teuthology.orchestra.run.smithi183:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T09:26:56.621 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T09:26:56.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T09:26:56.652 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T09:26:56.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T09:26:56.679 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T09:26:56.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T09:26:56.710 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T09:26:56.719 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T09:26:56.780 INFO:teuthology.task.internal:Duration was 653.671016 seconds 2024-09-17T09:26:56.780 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T09:26:56.845 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T09:26:56.845 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T09:26:56.848 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T09:26:56.850 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T09:26:56.886 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T09:26:56.890 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T09:26:56.893 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T09:26:57.353 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T09:26:57.353 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi003.front.sepia.ceph.com 2024-09-17T09:26:57.354 DEBUG:teuthology.orchestra.run.smithi003:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T09:26:57.378 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi077.front.sepia.ceph.com 2024-09-17T09:26:57.378 DEBUG:teuthology.orchestra.run.smithi077:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T09:26:57.403 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2024-09-17T09:26:57.403 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T09:26:57.429 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T09:26:57.429 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T09:26:57.432 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T09:26:57.445 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T09:26:57.500 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T09:26:57.500 DEBUG:teuthology.orchestra.run.smithi003:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T09:26:57.503 DEBUG:teuthology.orchestra.run.smithi077:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T09:26:57.511 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T09:26:57.701 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T09:26:57.712 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T09:26:57.712 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T09:26:57.738 DEBUG:teuthology.orchestra.run.smithi077:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T09:26:57.764 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T09:26:57.789 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T09:26:57.853 DEBUG:teuthology.orchestra.run.smithi003:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T09:26:57.856 DEBUG:teuthology.orchestra.run.smithi077:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T09:26:57.859 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T09:26:57.878 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = core 2024-09-17T09:26:57.879 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = core 2024-09-17T09:26:57.881 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2024-09-17T09:26:57.920 DEBUG:teuthology.orchestra.run.smithi003:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T09:26:57.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T09:26:57.952 DEBUG:teuthology.orchestra.run.smithi077:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T09:26:57.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T09:26:57.967 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T09:26:57.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T09:26:57.981 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T09:26:57.991 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T09:26:57.992 DEBUG:teuthology.misc:Transferring archived files from smithi003:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908293/remote/smithi003 2024-09-17T09:26:57.992 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T09:26:58.031 DEBUG:teuthology.misc:Transferring archived files from smithi077:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908293/remote/smithi077 2024-09-17T09:26:58.032 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T09:26:58.066 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908293/remote/smithi183 2024-09-17T09:26:58.066 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T09:26:58.101 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T09:26:58.102 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T09:26:58.104 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T09:26:58.106 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T09:26:58.150 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T09:26:58.161 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T09:26:58.161 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T09:26:58.240 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T09:26:58.240 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T09:26:58.244 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T09:26:58.246 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T09:26:58.256 INFO:teuthology.orchestra.run.smithi003.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 09:26 /home/ubuntu/cephtest 2024-09-17T09:26:58.258 INFO:teuthology.orchestra.run.smithi077.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 09:26 /home/ubuntu/cephtest 2024-09-17T09:26:58.261 INFO:teuthology.orchestra.run.smithi183.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 09:26 /home/ubuntu/cephtest 2024-09-17T09:26:58.262 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T09:26:58.295 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T09:26:58.351 INFO:teuthology.run:Summary data: description: orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} duration: 653.6710164546967 owner: scheduled_teuthology@teuthology success: true 2024-09-17T09:26:58.352 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T09:26:58.425 INFO:teuthology.run:pass